Mitigating Usability Risks:
The upcoming NMPA guidelines
in China

Mitigating Usability Risks in Medical Device Development, in the light of the upcoming National Medical Product Administration (NMPA) guidelines in China.

Over recent years, healthcare innovation in China has accelerated massively. From AI-empowered solutions and innovative devices and tools, to increasing connectivity and digitalisation, technology and design are transforming the experience of patients, carers and healthcare professionals for the better.

A crucial milestone in that journey will be the upcoming application of the NMPA’s ‘Guidelines for Technical Review of Human Factors Design of Medical Devices’, published in draft form two years ago. The acceptance of this guidance is now imminent, with significant consequences for how medical devices in China are designed and validated.

Once officially issued, this new guidance will place an obligation on medical device companies to recognise the potential impact of usability risks and conduct their design and development process accordingly. Any medical device product launching in China in the near future needs to consider this guidance so companies need to adopt the guidelines without delay, even for ongoing projects. This is because, inevitably, adopting to this approach will take some time, not only in terms of team mindset but also in terms of the transition required around process and resourcing. In this context, time is of essence.

How to mitigate the risks of summative usability tests?

As a critical part of usability validation, summative usability tests have been conducted in international markets for some time – following the recommendations of the US Food and Drug Administration (FDA) and the European Union Medical Device Regulations (MDR). For the majority of medical device companies in China however, they are still a relatively new tool in the design validation process and there is still a degree of hesitation when it comes to running these validation tests for new products.

Reluctance is partly driven by the fear of “failing” a summative test – i.e. the fear that the new process will lead to a design refinement or a redesign, with potentially huge commercial impact and a disrupted launch schedule. In reality, of course, it is impossible to “fail” a summative test as any learning is beneficial.

Although hesitancy is understandable, it is also misplaced. To adopt this process, medical device companies need to test more, not less, and run earlier and more frequent formative evaluations that can de-risk subsequent stages. Finding potential for improvement during development is a win-win, as fixing problems early saves money (recalls can be costly) and creates a better product – which in turn is likely to underpin a commercial advantage.

The main purpose of adopting a Human Factors Engineering (HFE) process for new product development is to mitigate risks relating to usability. So, how can we avoid finding issues at a point where they are difficult to fix (either just before a product release or post release)? The answer is simple – by engaging users more frequently, at the earlier stages of the design and development process. Formative evaluations can in fact identify ‘bumps in the road’ early, ahead of the summative evaluation. After that, there may be new learnings at summative / validation testing stage, but there will already be an established rationale around the design to work from.

Embracing imperfection in formative evaluations

Some manufacturers already carry out different kinds of formative evaluation, taking prototypes to the intended users to collect feedback and comments. These activities are typically associated with Usability Engineering (UE), and the information collected helps inform design iterations.

However, these formative activities could happen more frequently through the design and development process, with different fidelity of mock-ups or prototypes. Although it might feel uncomfortable or counterintuitive to present unfinished or partially functional prototypes to users, these are actually very useful to spark interactive discussions and co-creation. They are often a source of meaningful inspiration and provide a safe space to navigate and modify solutions before the design is frozen.

Crucially, any evaluation needs to encourage honest feedback.  Design and innovation always benefits from multidisciplinary contributions and the involvement of marketing and sales teams is important. This can be reflected in terms of the fidelity of the test setup (e.g. the extent to which a real-world context is simulated) and also the objectivity of the feedback (avoidance of bias). This also means that the purpose of the test has to be clearly communicated to participants, whether they are KOLs, leading surgeons, busy practitioners or patients, to ensure that quality is maintained. Even if usability testing sounds unfamiliar to some stakeholders, from our experience most people are willing to share their personal experience and feedback to contribute and improve products and therapies – i.e. for the greater good.

Never isolate a product from context during design and development

No medical device can be used in isolation. That’s why the clinical or home care context must always be taken into consideration in product development. R&D teams will always focus on the product itself and do their best to develop functionalities that meet product and system requirement specifications. But clinical practices can’t be neglected since they often determine the interface between the user and the product.

At PDD, we have often seen how products that worked well in a lab end up presenting significant issues when used in a clinical environment. It could be, for example, that the usage of a product might violate sterilisation rules, or that some tasks in the clinical setting might require collaborations between people that weren’t originally considered. Identifying and foreseeing these important influential factors is the beauty of early-stage user research, either through voice-of-customer interviews, ethnographic research and contextual inquires or expert reviews.

Don’t leave it all until the summative usability test

Of course, project teams might be tempted to stick to the project schedule and avoid what may be seen as time-consuming activities associated with user research. However, from our experience, running formative evaluations does not take long. The concern sometimes is about design iterations – if any risks are identified they might challenge an already tight development timeline and require work to be undone. No doubt, a cross-functional review of the risks and associated design refinements needs to be conducted following any form of user testing, but skipping formative evaluations inevitably leaves those risks hidden until they emerge at a later point. By then, it’s often too late and too costly to revise the design.

Safety is the baseline; a well-considered user experience is what makes the product outstanding

Although the regulatory requirements on HFE focuses on safety, user experience is what will make the product stand out. Well-considered human factors work contributes to user experience, will bring extra satisfaction to users and allow for differentiation.  During our work in clinical environments, we often see how users might not remember a product’s brand name but they clearly know which one is their favourite due to well-designed features or small usability details – like how easy the cap or packaging can be opened, given that they have to do the same thing numerous times every day. All these details are not invented in R&D’s labs, they are a direct result of the empathy with which we build the products and a human-centred approach to design

Conclusion

Whether it is FDA, MDR or NMPA, ultimately the goal of Human Factors Engineering is to support medical device companies in improving the design of devices to minimise potential use errors and resulting harm. Continuous efforts invested in HFE, from user requirement identification and generation, user interface design and formative evaluations will pave the way to alignment with the upcoming NMPA HFE guidance. This will help companies in China design medical devices that respond to the needs of patients and healthcare professionals, with an outstanding product experience to underpin their commercial success in domestic as well as international markets.