The combination of AI and radiologists has been found to increase the screening accuracy of radiologists for the detection of breast cancer
The combination of an artificial intelligence (AI) system and a radiologist provides better screening accuracy for breast cancer as demonstrated by a higher sensitivity and specificity according to the findings of a retrospective analysis by an international team of radiologists.
The use of screening mammography is designed to identify breast cancer at earlier stages as treatment will be more successful. Moreover, in recent years there has been increased interest in the use of AI systems and a recent study found that the use of an AI system outperformed all of the human readers, with a greater area under the receiver operating characteristic curve margin of 11.5% for screening breast cancer mammograms.
Nevertheless, a 2021 systematic review which considered the use of AI for image analysis in breast cancer screening programs concluded that the current evidence for AI does not yet allow judgement of its accuracy in breast cancer screening programmes, and it is unclear where on the clinical pathway AI might be of most benefit.
Other work that considered the role of AI for breast cancer screening suggested that an AI system can correctly identify a proportion of a screening population as cancer-free and also reduce false positives and therefore has the potential to improve mammography screening efficiency.
But what if an AI and radiologists worked together, so that the AI could initially triage scans and identify normal cases but those with suspected cancer and where there was diagnostic uncertainty, were referred to the radiologist? This was the question addressed in the retrospective analysis by the research team.
The system was designed so that the AI system would flag potential cancerous scans and where it was unsure about the diagnosis, for a second read by a radiologist. The team initially trained the AI system using an internal dataset and then used an external data set and compared the interpretation with that of a radiologist.
The performance of both the AI and radiologists was assessed in terms of sensitivity and specificity and the test sets contained a mix of both normal and cancerous scans.
AI and radiologists combined performance
For the external data set the radiologist had a higher sensitivity (87.2% vs 84.6%, radiologist vs AI system) and specificity (93.4% vs 91.3%) and in both cases this difference was statistically significant (p < 0.001 for both).
However, when the AI and radiologists worked together, the radiologist’s sensitivity was 89.7% and the specificity 93.8%. In other words, the combination improved both sensitivity and specificity. The authors calculated that this corresponded to a triaging performance, i.e., the fraction of scans which could be automated) of 60.7%.
Based on these findings, the authors concluded that their system leverages the strength of both the radiologist and the AI system and had the potential to improve upon the screening accuracy of radiologists.
Citation
Leibig C et al. Combining the strengths of radiologists and AI for breast cancer screening: a retrospective analysis Lancet Digit Health 2022
