Accelerated Access Collaborative – using our evidence standards framework in the Artificial Intelligence in Health and Care Award
Case study
At NICE, we drive innovation into the hands of health and care professionals to enable best practice. The recent growth in digital health technologies has the potential to revolutionise health and social care delivery. By placing a significant focus on digital health, we can help to address some of the UK’s most significant health and care challenges, while stimulating a thriving UK life sciences sector. In this case study, Emma Hughes explains how the Accelerated Access Collaborative within NHS England has incorporated the NICE evidence standards framework (ESF) for digital health technologies into their Artificial Intelligence (AI) in Health and Care Award.
“For practitioners, patients, and the public to benefit from technological advances and put their trust in them, we must make it easier for developers and adopters to understand what good levels of evidence for digital health technologies look like.”
Jeanette Kusel, director of scientific advice at NICE
“Having an ESF that's regularly refreshed will help to ensure that the evidence we generate today will meet the system requirements of tomorrow.”
Emma Hughes, senior manager for medtech and digital at the Accelerated Access Collaborative and evaluation lead for the AI in Health and Care Award.
About the framework
Our ESF describes the evidence requirements of different types of digital health technologies. It helps innovators understand what kind of evidence they need to generate to prove the effectiveness and value of their product when selling to the NHS. It also helps commissioners to make more informed and consistent decisions when buying digital health technologies.
About the AI in Health and Care Award
The Accelerated Access Collaborative runs the AI in Health and Care Award in partnership with the NHS AI Lab and the National Institute for Health Research. It is making over £100 million available to accelerate the testing and evaluation of the most promising AI technologies which meet the strategic aims set out in the NHS Long Term Plan.
Emma Hughes uses our evidence standards framework in her role as evaluation lead for the AI in Health and Care Award.
Emma Hughes uses our evidence standards framework in her role as evaluation lead for the AI in Health and Care Award.
“My job is to arrange evidence assessments for our most mature (phase 4) winning technologies,” begins Emma. “These already have market authorisation but insufficient evidence to merit large-scale NHS commissioning or deployment. I commission the NICE Scientific Advice team to conduct medtech early technical assessments of these phase 4 technologies and they refer directly to the ESF during this process. By identifying evidence gaps and key research questions, they provide feedback on the additional evidence the company needs to generate to meet the full set of ESF standards.”
A good example of how the ESF has benefited Emma in her role relates to a winning technology that analyses images of skin lesions to help detect skin cancer. “One of our evaluation partners was planning to run a cost-benefit analysis to assess the technology’s value to the NHS,” Emma reveals. A cost-benefit analysis is one of the ways in which economic evaluation of technologies can be carried out. The costs and benefits are measured using the same monetary units (for example, pounds sterling) to see whether the benefits exceed the costs.
But the ESF made clear that a cost-utility analysis was required instead. In this case, the benefits are assessed in terms of both quality and duration of life and expressed as quality-adjusted life years (QALYs).
“Having the framework to refer to, from such a well-respected and trusted organisation as NICE, helped to create a shared understanding between ourselves, the company and the independent evaluation partner on what kind of analysis needed to be done,” says Emma.
An updated framework
NICE recently added some important updates to the framework. It now:
- is clearer and easier to use
- specifies evidence requirements for data-driven technologies such as AI with adaptive algorithms
- aligns classification of digital health technologies with regulatory requirements.
In the fast-paced environment of digital health, Emma welcomes NICE’s updated framework. “It’s a challenge for our evaluation partners to future-proof their assessments,” she acknowledges. “Regulatory requirements and evidence standards are rapidly evolving. Having an ESF that's regularly refreshed will help to mitigate that risk, ensuring the evidence we generate today will meet the system requirements of tomorrow.”
Emma is also pleased that the ESF now includes deployment considerations for evidence-generation programmes. “Standards around communication strategies and training plans should help to ensure staff use the digital health technology correctly,” she enthuses.
Sound advice
For NHS commissioners making purchasing decisions about digital health technologies, Emma offers these words of wisdom. “Familiarise yourself with the 21 standards that make up NICE's evidence standards framework,” she says. “Also, if the developer has published evidence in a peer-reviewed journal, it can be a useful indication that they are open and transparent with their evidence.”
For digital health developers who are exploring how best to introduce their products to the NHS, Emma gives this advice. “The ESF contains helpful guidance on the evidence you need to generate. It also signposts you to other sources of information such as regulatory standards and best practice support that you should refer to. Use it to conduct a gap analysis of your evidence generation plans and make sure that you are meeting all of the requirements.”
Built with Shorthand