In 2016 Geoff Hinton, the British cognitive psychologist and father of artificial intelligence (AI) and machine learning, caught the attention of politicians by declaring that artificial intelligence, and machine learning, would replace radiologists. He could not have been further from the truth. It is clear from the recent presentations at ECR and RSNA that radiologists will become smarter and more efficient with the help of AI, but will not be replaced by it. Radiologists have been using AI for a long time now. Most radiology departments in the National Health Service (NHS) use voice recognition technology – which uses computer audition AI. The success of voice recognition technology is down to the integration with radiologists’ image reading workflow.
These are many types of computer vision AI that are being assessed for use in radiology including:
- Computer-aided anomaly detection – AI would be used for detection of abnormalities such as breast lesions, lung nodules on CT, lung shadows on chest X-ray, stroke on CT, haemorrhage on CT, fracture on plain X-ray, colonic polyp, pulmonary embolism detection on CT, etc
- Computer-aided simple triage – when an anomaly is detected, AI could be used to raise the priority of reporting in the worklist. This is being used for CT detection of haemorrhage in the head and prioritises reporting
- Computer-aided change detection – this form of AI would allow detection of change on serial studies, for example, multiple sclerosis, tumour progression, etc
- Image fusion or co-registration – these machine learning algorithms allow fusion of images between different modalities, for example, CT and MRI, etc
- Computer-aided classification (also called computer-aided diagnosis) – provides a possible diagnosis/nature of the lesion, or malignancy risk. It uses algorithms that take into account image features, location of lesion and radiomics (textural analysis by computer that is not detected by the human eye) to assess the risk of malignancy or the likely diagnosis. It is being used for assessing risk of malignancy in breast and lung lesions.
- Computer-aided segmentation – used for drawing around structures on the images. It can be used for drawing around tumours or even around normal structures for radiotherapy planning. It can also be used for display of angiograms from a CT or MRI study.
- Computer-aided quantification – this form of AI will count the number, size and volume of abnormalities such as lung nodules on CT. It can be used for assessing percentage involvement of lungs in interstitial lung diseases, quantification of bone age and degree of vessel stenosis. It can provide an emphysema index, osteoporosis score and calcium score.
- Computer-aided prediction – can be used for predicting onset of stroke, prognosis of disease, survival and response to therapy.
By quantifying the shape volume and texture of the hippocampus, AI maybe used for predict the risk of dementia.
Diagnostic accuracy of AI algorithms
It is really important for radiologists and patients to know how much we really rely on computer AI? Computer vision AI is a diagnostic tool. Like any diagnostic test, AI algorithms have both sensitivity and specificity. No algorithm will claim to be 100% accurate. Hence, it is really important, when doctors take decisions on management of the patient, or radiologists issue a narrative actionable report, that they are aware of the limitations (sensitivity and specificity of the algorithm used defining risk of malignancy or tentative diagnosis, for example, 95% sensitive and 80%specific).
Hence, when the output from computer vision AI is sent to PACS, or enterprise viewer, it is mandatory that the sensitivity and specificity of the algorithm used are displayed along with the images and AI markers. Display of sensitivity and specificity of AI algorithms for the front line doctors and reporting radiologist is essential for patient safety.
The accuracy of these algorithms is very much dependent on the type of algorithms used, and also the acquisition parameters applied by the modality. If the algorithm is to be accurate, it is really important that the acquisition parameters are standardised prior to application of the algorithm. Hence, many of the computer vision AI algorithms need to be integrated with the modality. For example, if there is a requirement for lung nodule detection on CT chest protocol, the scanner should automatically reset the acquisition parameters to optimise lung nodule detection.
Artificially intelligent machines and scanners
Current X-ray machines, CT scan and MRI scanners produce images only and will be replaced by intelligent machines and scanners in the future. All digital radiography machines will apply a chest shadow algorithm and produce images along with markers for shadows. The sensitivity and specificity of the marker detection will be displayed along with the images. Similarly, digital radiography machines have fracture analysis. The sensitivity and specificity of the output will be sent to PACS along with fracture markers. Likewise, CT scanners will have lung nodule detection algorithms applied. CT scanners will also output the emphysema index, osteoporosis index and calcium score along with their sensitivity and specificity for calculation for each of these algorithms.
Artificially intelligent PACS
Currently PACS is only capable of displaying images as sent by modalities. However, in the future, they will also be expected to display CAD markers. PACS vendors will also apply some computer vision algorithms. These include image fusion or co-registration. This will enable radiologists to fuse images from different modalities while automatically modifying the zoom, slice thickness and the acquisition parameters. PACS vendors may apply computer aided segmentation algorithms for anomalies such as liver metastases (for comparison and follow-up).
Future PACS reading workflow
Currently radiologists are not supported by computer vision AI. However, the rapid pace of development is afoot and the radiologist reporting workflow is going to change. Currently radiologist read images without any CAD markers. In the future, many forms of detection markers along with quantification and classification of anomalies will support radiologists reading workflow. We will need to learn how to issue narrative actionable reports in the context of AI, while understanding the sensitivity and specificity of the algorithms applied.
We will need to be able to discount anomalies when the computer gets it wrong. We will need to provide more human understandable narrative report from the scientific jargon provided by computer algorithms. Once again radiologists working patterns are about to change enormously. Computer vision will help radiologist detect subtle masses on mammograms, detect lung nodules on CT, fractures on plain X-ray, shadows on chest X-ray, etc. Computer vision will also provide a malignancy index for breast lesions and lung nodules. Outputs such as emphysema index, osteoporosis score and calcium score will be generated by CT scanners.
Conclusions
Computer vision will aid radiologists in producing narrative and actionable reports. These computer vision algorithms will provide great decision support for radiologist reporting. It is envisaged that the modalities such as digital radiography machines, CT scanners and MRI scanners will output intelligent information along with images. Intelligent machines will replace the current machines. It is also envisaged that current PACS systems will be replaced by intelligent PACS systems. Radiologists will learn to understand these computer-generated scientific outputs to produce human readable actionable reports to support patient management decisions.
