Blog: The NHSX buyers checklist for AI – how does our breast cancer screening AI, Mia, measure up?
The COVID-19 pandemic has upped the pace at which healthcare systems are adopting new technologies. It makes it even more inevitable that AI will play an increasingly important role in the healthcare challenges we face but accelerating its adoption comes with a special responsibility: patient safety must always come first so AI solutions must demonstrate they are safe and efficacious for all patients and clinical settings.
That’s why the NHSX buyers checklist, published last month, is so timely – it gives those in charge of procurement a list of critical questions they need to ask when evaluating novel AI technologies. It’s the latest in a series of documents from the NHS on AI technologies – they published their Code of Conduct in September 2018 (see our response here) and the NHSX report on AI: how to get it right in October 2019. We were mentioned in the foreword:
“In the near future, this kind of tech (Mia™) could mean faster diagnosis, more accurate treatments and ultimately more NHS patients hearing the words ‘all clear’.”
Matt Hancock, Secretary of State and Baroness Blackwood, Minister for Innovation
Does Kheiron measure up?
We’ve used the buyers guide as a benchmarking exercise to check that our solution, Mia, will measure up in the marketplace.
The questions in the buyers guide fit into three categories – does the solution work? Are the right processes in place for its adoption? And is the organisation ready to adopt it? We looked at each in turn:
Does it work:
Is AI the right solution for the type of problem you need to solve? Can this product do what it claims it can? Is it safe for patients?
In short, for Mia, we believe the answer is yes. Mia was designed together with radiologists to solve a critical problem – the radiologist workforce crisis in breast cancer screening. At Kheiron we always start with a problem that needs solving and one where AI can add actual value to a healthcare system. That’s why Mia is designed to relieve potential workforce burnout and can make the most impactful recall/no recall decision in breast screening while still having a radiologist in the workflow.
We believe we are pioneers in developing the clinical rigour required to ensure AI is safe and Mia offers a high quality and consistent performance for any woman. Mia has been rigorously tested in a clinical setting – firstly in a trial back in 2018 with results presented at a breast cancer conference. The results led to us gaining CE clearance for Mia. Secondly in a multi-site, multi-hardware independently validated trial on images representing a real world screening population in partnership with several NHS screening service providers.
These large-scale retrospective studies, overseen by clinical PIs and an independent contract research organisation (CRO) will be followed by prospective monitoring that happens in the background to build clinical acceptability, followed by live deployment.
We also used real world screening data for training, not a curated dataset – to better replicate the real world when assessing our solution. It’s a fundamental belief at Kheiron that in order for solutions to be safe and trusted by patients and clinicians, they must be developed and tested on data that represents the diversity of the real world.
Are the right processes in place:
Can this tech be procured through a transparent, fair, competitive process? Does this product meet regulatory standards? What agreements should you put in place to protect any IP generated by use of the product?
We are in slightly uncharted territory with these novel solutions so we need to work together to build the right safety nets and procedures. Our approach is to have close contact and ongoing conversations with early adopters.
For example, we have engaged closely with the Care Quality Commission (CQC) as a partner in their machine learning regulatory sandbox.
We also need to make sure we are using well established procedures and processes. For example, our trial was approved by PHE and MHRA and we have TUV as our notified body.
We are proud that Mia is a CE marked device within Europe (EU) and our claims are audited annually by TUV to ensure integrity, regulatory compliance and legal responsibility to sustain our business success.
We are always looking for early adopters who want to help shape how these new technologies are deployed in the NHS so if that sounds like you and you have a breast screening service, get in touch.
Is the organisation ready to adopt it?
Are users of the product primed to use it? Can you manage the maintenance burden of this new technology? Will your existing systems work effectively alongside the new technology to ensure a clear and reliable workflow? Do you have the necessary storage and computing requirements? What information sharing and data protection protocols would need to be in place to comply with organisation governance?
It’s refreshing how many points in the guide remind us that ‘adopting’ is a two way street. Any solution must be built to solve a problem and fit into the real world. An algorithm that is qualified as an ‘independent reader’ in a lab is one thing but is it of any practical use? That’s why Mia is developed with radiologists, for radiologists.
When it comes to the details, Kheiron offers a range of support services to any NHS Trust deploying Mia to ensure the solution is adopted. Mia comes with a team to ensure customer success. We will work closely with any Trust on all the activities they require to get clinical safety sign-off. We also already have a lot of experience working closely with information governance teams at Trusts from our UK government funded grant delivery projects.
And in our experience conversation is key – every NHS Trust has different needs so it’s vital we have clear and collaborative communications. This will help make sure Mia is solving their most critical problems and giving every woman in the UK a better fighting chance against breast cancer.