An artificial intelligence (AI) algorithm trained with a large set of mammography data outperformed radiologists when it came to identifying breast cancer in a new study, but it also helped them improve their own performance, according to research published online February 6 in Lancet Digital Health.
The findings offer further evidence that AI can be an effective aid for radiologists in mammography interpretation, wrote a team led by Hyo-Eun Kim, PhD, of AI software developer Lunit. Dr. Hak Hee Kim of the University of Ulsan College of Medicine in Seoul, South Korea, and Dr. Boo-Kyung Han of Sungkyunkwan University School of Medicine in Seoul also served as co-lead authors.
"The significant improvement in radiologists' performance when aided by AI supports application of AI to mammograms as a diagnostic support tool," the group wrote.
Mammography remains the most widely used breast cancer screening modality, despite the use of additional technologies such as tomosynthesis, ultrasound, and MRI, Kim and colleagues noted. However, mammography has its limitations, which researchers have sought to address using AI in various forms, starting with computer-aided detection (CAD) 20 years ago and progressing to algorithms based on machine and deep learning that are increasingly common today.
In their study, the researchers developed and validated an AI algorithm (Lunit Insight MMG, Lunit) using 170,230 mammography exams collected from five facilities in South Korea, the U.S., and the U.K. Of these, 36,468 were malignant, 59,544 were benign, and 74,218 were normal.
For the study, the group selected 320 mammograms (160 malignant, 64 benign, and 96 normal) and enlisted 14 radiologists to assess the likelihood of malignancy, location of malignancy, and necessity for recall -- first without then with the use of the AI algorithm.
The algorithm demonstrated better performance than the radiologists in analyzing the mammograms, but it also improved their performance, the investigators found. All results were statistically significant.
| Breast cancer detection performance of AI algorithm compared with radiologists | |||
| Performance measure | AI algorithm | Radiologist reader, no AI assistance | Radiologist reader, AI assistance |
| Area under the curve (AUC) | 0.94 | 0.81 | 0.88 |
| Sensitivity | 88.75% | 75.27% | 84.78% |
| Specificity | 81.87% | 71.96% | 74.64% |
The group found the AI algorithm was more sensitive than radiologists in identifying the following:
- Masses: 53 (90%) compared with 46 (78%) out of 59 detected
- Distortions/asymmetries: 18 (90%) compared with 10 (50%), out of 20 detected
- T1 cancers: 73 (91%) versus 59 (74%) out of 80 detected
- Node-negative cancers: 104 (87%) versus 88 (74%) out of 119 detected
The study demonstrates that a large dataset makes for a robust AI algorithm for detecting breast cancer -- and offers a helpful way to support radiologists in their mammography workflow, the group concluded.
"The AI algorithm we developed with large-scale high-quality data showed better diagnostic performance than radiologists in breast cancer detection from mammograms," the authors wrote. "More importantly, the diagnostic performance of radiologists was significantly improved with the assistance of AI. This result shows that AI can be used as an effective diagnostic support tool for breast cancer detection."
