Using a mammography on-line self-assessment, less than half of Italian radiologists were able to successfully completed the test.
Detection of breast cancer requires that mammography scans are interpreted correctly and thus guidelines on screening require that robust quality control and quality assurance are in place. Although an assessment of performance against standards can be achieved through audits, such audits offer little insight of the diagnostic skills of individual radiologists. The use of on-line self-assessment, using pre-diagnosed mammograms, offers a means of identifying any knowledge gaps and could serve as an important educational tool.
An on-line self-assessment tool was adopted by the Italian Society of Medical Radiology in 2018 as part of a national on-line self-assessment program for interpretation of mammograms and the results have published. The test set was a collection of 24 pathology-confirmed cancers combined with 108 mammograms which were reported on as negative after double reading. All cases were acquired with full-field digital mammography and came from women aged 50 – 69 years, who represent those at average risk of breast cancer. For the assessment, the society determined a pass threshold for sensitivity of 62% (amounting to at least 15/24 cancers correctly identified) and 86% specificity (93/108 negative mammograms correctly identified). The on-line self-assessment was posted on a dedicated website and upon registration, radiologists provided information on demographics such as age, gender, place of work (public or private sector), years of breast imaging experience and a qualitative self-judgement of mammography interpretative skills as “beginner”, “average” and “expert”. As with usual mammography practice, the radiologists were required to submit a dichotomous diagnosis (i.e., positive/negative) for every case. In addition to reporting on the radiologist’s success in the test, the authors employed regression analysis and odds ratios to identify the most important predictor variables for diagnostic accuracy.
Findings
A total of 685 radiologists registered for the on-line self-assessment and 49.9% (342) with a mean age of 46 years (69% female), completed test. Among those completing, two-thirds (64%) self-judged their interpretative skills as “average” and respondents had a median of 8 years breast imaging experience although 38.3% reporting having more than 10 years’ experience. Participants reported a median number of 1501 mammographic interpretations per year and the majority (68.7%) worked in the public rather than private sector.
When examining the proportion who successfully completed the assessment, only 28.7% of radiologists (98/342) passed on their first attempt. After an initial failure, 138 of the remaining 244 radiologists, re-took the test of whom, only 35.5% (49/138) passed. In fact, the authors reported that overall, only 44.2% (151/342) of radiologists who completed the on-line self-assessment were successful.
Using regression analysis, a significant association for diagnostic accuracy was found only for the assessment of > 3,000 mammograms per year compared to < 1,000 (Odds ratio, OR = 3.88, 95% CI 1.07 – 14.14, p = 0.04) and working the in public rather than private sector (OR = 1.65). Other variables such as age, self-judged interpretative skills and years of breast imaging experience had no significant effect.
The authors concluded that their study suggested that breast imaging experience does not guarantee diagnostic accuracy in screening reading. They also noted that their on-line self-assessment test could be included as a criterion for the accreditation process of breast units.
Citation
Brancato B et al. Mammography self?evaluation online test for screening readers: an Italian Society of Medical Radiology (SIRM) initiative. Eur Radiol 2021
