The Risks of Risk Analysis

Risk analysis is now embedded in so many quality standards that its inclusion is almost mandated in project plans. But, is risk analysis everything it is deemed to be? At PDD, we have used Failure Mode Analysis methods and tools for over 25 years – it’s perhaps a good time to challenge some of the underlying beliefs behind these systems.

At PDD our use of risk analysis tools might be unorthodox since we start to use them early in a project and then progress their use as the design develops.  Our Fault Tree Analysis (FTA) and Failure Mode Effect and Criticality Analysis (FMECA) tools are, in the first instance, blank templates to be populated by specified requirements, design functions and then potential risks and actions. We don’t drive the projects with prepopulated risk analysis tools, although they do include an essential requirements checklist. Instead, we tend design to requirements and functions and then assess the fitness for purpose of designs using the risk analysis methods and tools. We find this a more creative and lean approach.  Also creative is the negative brainstorming method we apply for the risk analysis sessions to anticipate failure modes.

Selecting the right time to employ the right risk analysis tool is also important.  When design is formative, we prefer to use FTA (aka top-down FMEA) – appraising concepts and early designs against headline requirements, to coincide with key phase milestones such as completion of design intent. Only later, throughout engineering and post-prototype do we employ the more involved FMECA tool, assessing assemblies against their overall requirements and components against their specific functions.  Some would argue that the FTA and FMECA should meet in the middle with equal risks and scores.  However, as we employ them at different times in the product development pursuing that level of alignment can be a futile, laborious effort that distracts from the more important work of reducing risk.

Managing expectations around risk.

It is also important to acknowledge that, despite the comprehensive use of risk analysis tools, risks can still occur. From decades of experience working on risk assessments at PDD we have observed that there are 4 main reasons why risk might not be mitigated despite the use of risk analysis:

Firstly, are the ‘side-winding’, unpredictable events that no-one foresaw that can emerge often during early models or prototypes. These are the unexpected Black Swan events (Willem de Vlamingh).  From the point they emerge they become part of the FMECA for management but many of the design opportunities to remedy these risks inexpensively might have already passed. Employing risk analysis early is still the best way to avoid these risks and the escalating expense of change to rectify them. The existence of Black Swans is also one of the strongest arguments for early POP (proof of principle) models and prototyping.  Thankfully, most of our client agree.

Secondly, and potentially most damaging, are those risks that are known but go unreported and occur outside the format of the FMECA.  On many projects, there is an implicit weakness in the concept which might go unspoken because the project owners do not believe it. In extreme cases this can go as far as ‘The Emperor’s New Clothes’.  It might be too big, confusing, difficult to use, costly, etc but to mention this is taboo, so it is unlikely to be admitted to the FTA or FMECA. In the business world, such risks are known as Grey Rhinos (Wucker 2016)- those long-known problems, debated and understood but largely ignored.  Eventually if correct they will damage the project outcome.

Thirdly, but thankfully uncommon, is the belief that use of the tool alone will remove risk.  The tendency is then not to apply it sufficiently or with the gravitas it deserves.  In a sense, it becomes a tick box exercise without the necessary rigour to make it effective at reducing risk and making the design work.  To prevent this from happening, risk analysis must be taken seriously, and carried out by teams that have extensive experience and a broad perspective to be able to best identify the risks.  At PDD, we use the most experienced managers to review and lead FMECA sessions and invite our clients to take an active part in the sessions.  To continue with the animal theme, the risks that occur when risk analysis is not taken seriously enough make me think of Hyenas,  they might be laughing but nothing is funny

Lastly, and not least, it is important to realise is that the scores used in a FMECA are still based on collective opinion.  It is simply a ‘belief-based system’ that can suffer from groupthink. An overly strong leader of the assessment can influence the group’s score.  If inadequate scoring happens, then the risk won’t receive an adequate mitigating action.  Conversely, where risks are overly scored, problem  that are phantoms will detract valuable effort away from solving real problems. s a result of human belief, groupthink, or persuasion, some of the scores and risk-mitigating actions will then be undetectably out of proportion – Giraffe-like, if you will.

So, given these 4 reasons above, are the risk analysis methods still effective? From our decades of experience at PDD, the answer is ‘yes’ – there would be many more risks without them.  Their effectiveness is demonstrated and evidenced by all the mitigating actions, reduction in risk scores and avoidance and elimination of physical problems each time we have used and reused the tools on projects.

No process in the world can predict, by its very definition, the rare Black Swans. But an experienced team that takes risks analysis seriously should be able to keep the Grey Rhino, Hyenas and Giraffes managed to an optimum level. Our risk analysis tools are simple and accessible and with the right level of expertise and training are effective at managing the core risks throughout the project from concept to post-tool engineered solution.