Health Technology Wales - using our evidence standards framework in topic selection
Case study
At NICE, we drive innovation into the hands of health and care professionals to enable best practice. As part of this work, we’re placing significant focus on digital health over the next 5 years. By supporting the Government’s life sciences vision, we can help to address some of the UK’s most significant health and care challenges, while stimulating a thriving UK life sciences sector. In this case study, Dr David Jarrom explains how Health Technology Wales has incorporated our evidence standards framework (ESF) for digital health technologies into its topic selection process for technology appraisals.
“NICE is uniquely placed to help the UK health and care system release value from digital and artificial intelligence (AI) technologies, while helping to mitigate risk.”
Harriet Unsworth, technical adviser for digital health at NICE
“I welcome this updated ESF and am excited to begin using it in practice.”
Dr David Jarrom, principal researcher at Health Technology Wales
About the framework
Our ESF describes the evidence requirements of different types of digital health technologies. It helps innovators understand what kind of evidence they need to generate to prove the effectiveness and value of their product when selling to the NHS. It also helps commissioners to make more informed and consistent decisions when buying digital health technologies.
About Health Technology Wales
Health Technology Wales supports a national approach to the identification, appraisal and adoption of health and social care technologies in Wales. Their remit covers any technology or model of care in health and social care that is not a medicine.
“Implementing NICE's ESF within Health Technology Wales is a key part of my role,” begins David. “I also promote the framework to local commissioners, supporting its use within the Welsh health and care system.”
An updated framework
NICE recently added some important updates to the framework. It now:
- specifies evidence requirements for data-driven technologies such as AI with adaptive algorithms
- aligns classification of digital health technologies with regulatory requirements
- is clearer and easier to use.
David welcomes this updated version and is particularly pleased that it incorporates evidence requirements for AI. “Decision-making in this rapidly growing area is something everyone is grappling with,” he states. “Up until now, we’ve had little support in this area. Health Technology Wales truly welcomes this update to the ESF from NICE.”
Dr David Jarrom, principal researcher at Health Technology Wales, welcomes our updated framework.
Dr David Jarrom, principal researcher at Health Technology Wales, welcomes our updated framework.
Mitigating risk
Health Technology Wales uses the ESF to assist with technology appraisal topic selection. “When we are notified of a new digital technology, we use the ESF to help us classify it by function,” David explains.
Classifying digital health technologies in this way allows them to be put into evidence tiers based on the potential risk to users. The evidence level needed for each tier is proportionate to the potential risk to users from the digital health technologies in that tier.
“Once classified, we can refer to the ESF’s evidence tables to see the evidence requirements for that specific tier,” says David. “Then, we look to see if that evidence exists. This guides our view on whether the technology is mature enough to go forward for appraisal or not.”
Functional classification is not without its challenges and some technologies evade concrete classification. In these cases, the ESF recommends classification in the highest applicable evidence tier. “Having the framework and guidance to refer to, from such a well-respected and trusted organisation as NICE, really helps to provide shared understanding and clarity for both innovators and commissioners,” enthuses David.
Striking the right balance
The evidence underpinning digital health technologies is often not as mature as that of traditional medical devices. If we set evidence standards too high, we could create barriers for patient and system access to potential innovation. But, if standards are too low, it will be difficult to identify those products that are transformational for patients and offer the most value to the system.
A good example of this is Health Technology Wales’ recent guidance on electronic blood management systems for blood transfusion. Traditionally a manual process, electronic blood management systems use barcodes to make sure the correct protocol is followed and patients receive the right blood. “Evidence around the benefits of such systems is challenging to interpret,” David acknowledges. “The trial designs we looked at were not as robust as we’d normally see.” But the ESF provided David and his colleagues with reassurance that the evidence presented for this type of digital product was acceptable and that they could use it to support their decision-making. Guidance from Health Technology Wales will now inform how this technology is used across the Welsh health system.
Words of wisdom
For NHS commissioners making purchasing decisions about digital health technologies, David believes that when it comes to the ESF, practice makes perfect. “I’d recommend going through it a few times, working through some examples," says David. "The more you use it, the easier it will become.”
For digital health developers who are exploring how best to introduce their products to the NHS, David concludes by offering this sound advice. “Look to the ESF,” he asserts. “Think about where your product fits and what evidence you need to generate to satisfy a body like Health Technology Wales or local commissioners.”
Built with Shorthand