Getting to the root of the problem!
Published: 04 July, 2007
Andy Mellor, senior engineer with Stockport based AV Technology, talks to PWE about Root-Cause Failure Analysis (RCA) in a series of questions and answers.
Predictive maintenance and condition monitoring programmes significantly improve equipment reliability and plant productivity, while pouring scorn on the old adage 'if it ain't broke don"t fix it’. However what happens when things do go wrong; when despite ‘best efforts’ equipment breaks down through unforeseen critical component failure?
Root cause analysis is now becoming well established as a structured method for investigating equipment failure and coming up with a sustainable solution.
PWE: Briefly explain what Root Cause Analysis (RCA) is all about.
AM: Simple. It’s about a logical, disciplined approach to solving chronic problems that are hurting profits throughout UK Industry. It’s about dealing in facts - once you have uncovered the facts you can start developing solutions that address the real causes of problems.
PWE: That suggests that you think RCA has wide application?
AM: Yes, I believe, without a shadow of doubt, that RCA methods should be used in every factory, plant and facility in the UK. I mean, what are the alternatives? If you don’t deal in facts, what do you use? Traditionally we’ve had ‘Witch-hunts’ where we’ve assigned blame – that just causes people to keep quiet about mistakes for fear of punishment. It drives the problems underground – they’re still there, hurting profits. Using experience – I’m not decrying experience, especially now I’m getting old enough to have some, but we have to face the fact that we still have recurring chronic problems in industry despite the involvement of experienced craftsmen and engineers. Furthermore, in industry we have an ageing craft workforce and a skills gap caused by the failure to invest in apprenticeships in the 90’s. Guesswork – I think not! We need to uncover the facts, otherwise our solutions will end up addressing the symptoms, and we won’t get the benefits we’re looking for.
PWE: Every factory?
AM: Yes – look at Japanese approaches such as TPM (Total Productive Management). RCA is fundamentally a part of TPM through Kaizen and Poka-Yoke (Mistake proofing). Anyone implementing Lean Manufacturing strategies should be using RCA because chronic problems are definitely sources of waste. I’ll give the sceptical reader a parallel example – If you had a spate of similar industrial accidents resulting in minor injuries, on health and safety grounds you’d investigate and spend whatever resources were necessary to eliminate that problem. Your duty of care to your employees would demand that. Your duty of care to your shareholders should demand that you respond appropriately to chronic equipment issues. That’s good management – if you’re not trying to improve, you’re going backwards.
PWE: What is the process you go through when conducting an RCA? – give me an overview.
AM: Ok, I’ll start with the aims. The aim of an RCA is to identify the underlying causes of a problem and to develop solutions to address these problems so that the problem is permanently and comprehensively dealt with. To do this we need to get to the facts, not only about the physical causes, but about the underlying human causes. Tackling these causes is truly pro-active, because we will also be preventing future problems in other areas – I’ll give an example of that later. The first step in RCA is to define the problem, the event that is occurring. Then all the possible causes and factors contributing to the problem are listed. This can take some work and may need to draw on the expertise of a wide group of people. It’s very important not to jump to conclusions too early. Obviously there are various tools and approaches you can use here to order and assess the evidence. Our approach to RCA isn’t dogmatic in that sense.
Each possible cause is then examined in turn. Anything that we can’t disprove has to be considered as being a possible ‘immediate cause’. This process of listing potential causes and trying to knock them over is then repeated with the immediate causes – and so on, and so on until you have drilled down the chain of cause and effect to the root physical causes. We can then develop a technical solution to the problem – there are also a number of conceptual tools that can be useful here.
It’s then important to carry on and identify the acts and omissions of humans which contributed to the physical roots and also to identify the causes of these acts and omissions – be it lack of training, poor procedures or standards, attitudes etc. Often there are cultural reasons that problems are tolerated. When management start addressing these issues, the most significant paybacks start.
PWE: This sounds like a tall order for one person to accomplish.
AM: Yes – with a small problem, one person with the right skills could dig down to the physical roots, but on bigger RCA’s you need to work in teams and you need to be able to call on outside expertise. A lot of the work is in collection of the relevant data. It’s important to have a ‘no-blame’ or a ‘just and fair’ culture, otherwise vital information about mistakes can be swept under the carpet. To address the latent, human roots you need management backing and support. Team working also has an important part to play in this. Since teams own the solutions they develop, the solutions are more likely to be implemented and followed through.
PWE: What do you think are the key strengths for an RCA team leader?
AM: Above all a capacity for logical thought, then tenacity and persistence, tact, the willingness to admit ignorance and ask questions, open mindedness, a good general understanding of engineering and knowing where to go for specialist back-up and help. AVT has in-house expertise in diagnosis and fault-finding on rotating equipment including its installation, operation and maintenance, lubrication management and a good working knowledge of Tribology and engineering structures, but we also have an extensive network of contacts for specialist services. It’s very important to know your own limitations as an analyst as you will always be forced to look for answers beyond your core expertise. This is part of the excitement and challenge.
PWE: Can you give an example of RCA in action?
AM: In January 2006 we got involved in the investigation of severe wear of a drive pinion gear on a twelve-head carousel keg cleaner at a UK brewery. AVT were asked to investigate the cause of the keg pre-cleaner pinion failures as part of a proactive response supporting their Condition Monitoring Programme being carried out on the site.
The pre-cleaner had been commissioned in 2005 and at the time of the problems had been operational for only eight months, operating 160 hours per week under very arduous conditions, including caustic washdown.
The analysis identified specific deficiencies in the lubrication, manufacture and choice of material for the gears, potentially exacerbated by questionable gear alignment. Ingress of contaminants from caustic washing was excluded.
We found out that, when the keg pre-cleaner was installed, the brewery requested the original grease to be changed to a food grade equivalent. Investigation showed that the OEM maintenance manual recommended a lubricant for the gear teeth which was withdrawn from the UK market in the 1990’s. The failure to replace the OEM recommended lubricant with a suitable alternative was a significant contributing factor to the rapid wear.
Our final report came up with a number of key recommendations involving all parties concerned. These focused on improving the lubricant type and lubrication regime for the gear teeth, improving the gear set’s resistance to smearing and introducing best practice installation standards and start up procedures.
In conclusion, using our report as the focus, we chaired a meeting with the client, the lubricant supplier and the manufacturer of the pre-cleaner. The fact based approach and our neutrality, took a lot of the heat out of the situation. All parties admitted mistakes had been made and the meeting was conducted in an excellent atmosphere. The manufacturer admitted to quality problems with the one of their two suppliers of the pinion and agreed to supply and install replacement pinions on each of the three carousels in the keg filling line on a free of charge basis. Structured lubricant trials were agreed and changes to the lubrication system planned into production windows. Looking at the deeper issues, the client is considering a change management system. The supplier could well examine the design process leading to the choice of material selection and surface finish in that application.
For further information please visit: www.avtechnology.co.uk