Regulating artificial intelligence (AI) has been adequately described as sailing into uncharted territory. Complex challenges await regulators, who are witnessing the development and implementation of a new breed of algorithms, capable of continuously learning and adapting. In radiology, this new breed is being used to assist a number of tasks, from image interpretation to report management. Reliance on personal data to develop, train, and test such algorithms, potential clinical ramifications of their implementation, and issues surrounding liability, serve only to muddy the waters for regulators in this particular sphere. A recent panel, assembled in the AIX Theatre at ECR 2019, sought to address some of the major challenges in the way of achieving effective regulation of radiological AI. That panel, entitled “Where to next? The Regulation of Radiological AI in Europe and Abroad”, embarked on an expansive discussion, which can be viewed for free on ECR Online.
“Is the current regulatory system fit for purpose: yes or no?” was the first question moderator Dr. Hugh Harvey posed to the diverse panel, consisting of Dr. Bibb Allen, Marina Codari, Abdul Hamid Halabi, Dr. Lawrence Tanenbaum, and Prof. Greg Zaharchuk. As Hugh Harvey elaborated, the current regulatory schemes operating in Europe and the USA – which recognise algorithms used in health as medical devices – implement a build and freeze approach: “You build your algorithm, you freeze that code, you then validate it on some unseen validation set and show the regulators”, leaving you with an approved but static product. However, as the panel discerned, this approach has a number of limitations when it comes to the ‘modern’ algorithms being applied to image interpretation, among other tasks.
Abdul Hamid Halabi described the shift in the way algorithms are being used, stating “The way algorithms are created today is very different from the way they were created in the past. While in the past we understood exactly and defined what the thing we are trying to look at is, today we’re [algorithms are] learning from examples… because you’re learning from examples, you never stop learning.”
Responding firmly to Hugh’s question, Bibb Allen stated that the current system is not fit to handle software as a medical device and that regulators, such as the FDA, recognise that. According to him, the difficult question facing regulators is the following: Is there a way to monitor the performance of algorithms in the clinical settings, so that they don’t need to be constantly exposed to new phases of approval as they develop? “Is the current regulatory system able to handle software as a medical device, in particularly AI? And the answer is no,” explained Allen. “The idea that we would run a full pre-market assessment with multi-centred trials that take years and years and millions of dollars to do then run it for the innumerable, I think it will be thousands and thousands of algorithms over the course of the next few decades that will be available to radiologists.”
Echoing those sentiments was Lawrence Tanenbaum: “How do you take a technology that progressively learns, that can improve incrementally, and keep [regulatory] approval going? That piece has not necessarily been addressed.” But, as he went on to assert, one benefit of current approaches to regulation is that algorithms are making it to the market fairly quickly, reflective of an ex-post regulatory approach. “These tools are getting approved, however, right. So to me, it’s almost like a license to use without expecting the reliability that we’d usually get,” said Tanenbaum. He explained further, “We’re not saying that it will be effective, we’re just letting you use it. At least for the foreseeable future, these things are going to come into our use and they’re not really going to perform the way we expect a CT scanner to perform or an MRI scanner to perform.”
A definitive question for regulators and, ultimately, users, will indeed be what level of reliability we expect from these tools.
The panel went on to discuss other issues and factors regulatory systems will need to comprehend. Marina Codari elucidated the importance of regulatory processes focusing on the data (labels) on which modern algorithms are trained – in addition to the algorithms themselves – and the intrinsic variability those labels hold. That need to focus on the accuracy of labels is why radiologists must be involved in the process, according to Abdul Hamid Halabi. Pathological variation across populations poses further difficulties for developers and regulators. As Greg Zaharchuk explicated, “Stroke in China is very different than stroke in the US. If you have an algorithm that’s going to predict stroke in the US, it’s very unlikely that it’s going to predict stroke well in China.” One possibility posed by the panel is the implementation of calibration phases to allow algorithms the opportunity to adapt to such variation.
Bias and the role of synthetic data, among other topics, were also touched upon as the discussion expanded. And, with the European Commission’s recent announcement of a pilot project, aimed at identifying whether their ethical guidelines for AI development can be implemented into practice, there may be a lot more regulatory issues being discussed during AIX at ECR 2020.
The ESR would like to thank all contributors to this panel.
Click here to view the full discussion at ECR Online.
- London/UK, Clinical Director, Kheiron Medical
- Former President of the American College of Radiologists
- Milan/IT, PhD Candidate in Integrated Biomedical Research presso Università degli Studi di Milano – State University of Milan
- Global Business Development Lead, Healthcare & AI at NVIDIA
- Washington D.C./US, Director of Advanced Imaging, Vice President and Medical Director Eastern Region at Radnet
- Professor of Radiology (Neuroimaging and Neurointervention), Stanford University
