The greatest threat to artificial intelligence (AI) uptake across the healthcare system is the ‘off switch’ if clinicians do not see the benefit of it, a new white paper has suggested.
If clinicians see the technology as burdensome, unfit for purpose or are wary of how it will impact on their decision-making, their patients or their licenses, they will not use it, the white paper said.
Published by the research initiative MPS Foundation, which is part of Medical Protection Society, the white paper looked at the impact of AI on clinicians, building on the results of its Shared CAIRE (Shared Care AI Role Evaluation) research project.
It also highlighted a concern that clinicians would absorb legal responsibility for AI-influenced decisions, even when the systems themselves could be flawed.
The report said: ‘We are at a turning point in the deployment of medical AI, as it moves from the margins to the mainstream of healthcare delivery. To unlock and realise AI’s potential for patients, it is essential that we implement and deploy AI technologies in ways that work for those using them – the clinicians.
‘If we fail to do so, we risk exacerbating the very challenges AI is supposed to address: clinician burnout, inefficiency and uneven patient experiences and outcomes.’
It outlined seven recommendations for the the Government and regulators to consider when developing new healthcare AI policy guidance:
- AI tools should provide clinicians with information, not recommendations to reduce any potential risk to clinicians and patients
- Revise product liability for AI tools before allowing them to make recommendations to avoid clinicians absorbing all of the liability even where the system is a major cause of an issue
- AI companies should provide clinicians with the training and information required to make them comfortable accepting responsibility for an AI tool’s use. This includes understanding its intended purpose; contexts it was designed and validated to perform in; and the scope, limitations and potential bias of its training dataset in order to deliver the best possible care to patients
- AI tools should not be considered akin to senior colleagues in clinician-machine teams and therefore clinicians should not always be expected to agree with, or defer to, an AI recommendation in the same way they would for a senior colleague
- Disclosure to a patient that an AI tool helped to inform a clinician’s decision should be a matter of well-informed discretion
- AI tools that work for clinician users need to be designed with them to deliver more benefits than burdens
- AI tools need to provide an appropriate balance of information to clinician users.
‘This white paper proposes recommendations which address the impact of decision-support tools on clinicians. The greatest threat to AI uptake in healthcare is the “off” switch, if frontline clinicians refuse to engage with technology they see as burdensome or unfit for purpose,’ the white paper said.
‘Given competing priorities for funding, political pressures and the need for good governance, it is more vital than ever to focus resources on AI solutions which will generate the most benefit. It is by understanding how AI can genuinely support clinicians that this benefit will most likely be achieved.’
Commenting on the white paper, Professor Gozie Offiah, chair of the MPS Foundation, said: ‘Healthcare is undergoing rapid change, driven by advances in technology that could fundamentally impact on healthcare delivery. The potential opportunities provided by AI are only limited by one’s imagination.
‘There are however real challenges and risks that must be addressed. Chief among those is the need for clinicians to remain as informed users of AI, rather than servants of the technology.’
She added: ‘If AI works well for clinicians, they are more likely to embrace and interact with it, and this will be vital in unlocking benefits to patients. We have written to the regulators and the Government minister to urge them to take on board these important recommendations.’
Along with MPS Foundation, the white paper was also a collaboration with the Centre for Assuring Autonomy at the University of York, and the Improvement Academy hosted at the Bradford Institute for Health Research.
It comes shortly after NHS Shared Business Services announced a new AI framework to optimise clinical workflow.
In October, the University of Cambridge appointed Professor Raj Jena as the UK’s first clinical professor of AI in radiotherapy, signalling a need for, and a commitment to, utilising AI in the fight against cancer.
Last summer, a survey from the Health Foundation found that NHS staff and patients were split on the use of AI in healthcare.
A version of this article was originally published by our sister publication Healthcare Leader.
