Computer-aided detection (CAD) technology improves sensitivity in breast MRI for both experienced and novice readers, but it does not significantly advance overall accuracy, according to a study published in the June issue of the American Journal of Roentgenology.
The study also showed no significant differences in the overall time it took the 20 readers -- 11 novice and nine experienced radiologists -- to interpret breast MR images with or without CAD.
The findings point to the need to develop educational programs on CAD to improve both overall diagnostic accuracy and the efficiency of breast MRI interpretation, wrote Dr. Constance Lehman, PhD, from the University of Washington and Seattle Cancer Care Alliance, and colleagues (AJR, June 2013, Vol. 200:6, pp. W683-W689).
Large datasets
Given that a typical breast MRI exam can include as many as 1,000 images obtained both before and after injection of contrast media, CAD software has been developed to assist radiologists in their interpretations and conclusions, the authors noted.
"Although these computer-aided detection systems are commercially available and widely used in breast MRI interpretation, little is known regarding the effect of the use of CAD systems on diagnostic accuracy and interpretation time for breast MRI," they wrote.
Prior to the study, the researchers conducted a short review of BI-RADS breast MRI terminology for the participating radiologists. The nine experienced readers had performed at least 100 breast MRI exams.
The 20 radiologists completed two reading sessions six months apart to minimize recall bias. Each session involved reading half of the 70 cases with CAD (CADstream, Merge Healthcare) and the other half without CAD. Among the 70 breast MRI cases, 27 had a benign outcome and 43 had a malignant outcome.
Image selection
Lehman and colleagues specifically chose MR images obtained through a variety of acquisition techniques at various institutions to gain a "representative real-world set of images on which to test the utility of the CAD system," they wrote.
The selection and order of MR images were randomized for each reader to further minimize the potential for bias. Color overlays and CAD tools were available and used at the discretion of the radiologist, while non-CAD interpretations included visual assessment.
Each radiologist was asked to identify any suspicious lesions and to rate the likelihood of malignancy. The readers were also instructed to provide an overall assessment of each case, as a whole, based on the most suspicious lesion identified.
Lehman and colleagues used a modified BI-RADS scale to measure readers' subjective assessment, along with a five-point probability of malignancy scale and a percentage probability of malignancy scale.
The readers' performances were measured using area under the receiver operator characteristics curve (AUC), and accuracy was compared across the use or not of CAD and the different experience levels.
MRI of a 47-year-old woman with invasive ductal carcinoma. Initial axial contrast-enhanced MR images without CAD (above) and with CAD (below) show enhancing mass in left posterior lateral breast that is more evident with CAD color overlay. Images courtesy of AJR.
Better for experienced readers?
In evaluating the results, CAD was found to improve sensitivity for both the experienced (AUC of 0.91 with CAD; 0.84 without CAD) and novice readers (AUC of 0.83 with CAD; 0.77 without CAD). The sensitivity increase was statistically higher for experienced readers (p = 0.01), according to the authors.
For diagnostic accuracy, AUC for novices was 0.77 without CAD and 0.79 with CAD. For the experienced readers, AUC was 0.80 without CAD and 0.83 with CAD. While there was an "upward trend," the differences were not statistically significant, they wrote.
There were no differences in specificity or positive predictive value. There was, however, a difference in negative predictive value among the experienced radiologists when reading with and without CAD (0.81 versus 0.70, respectively).
In evaluating the time it took readers to complete each MR image interpretation within a session, there were no significant differences with or without CAD among experienced and novice readers.
MR images with abnormal findings interpreted with CAD took an average of 1.18 minutes longer to evaluate. The researchers attributed the slight time increase to the "learning effect" radiologists experienced in using CAD.
"As the use of breast MRI with CAD increases, more attention to the potential contributions of CAD to the diagnostic accuracy of MRI is needed, and additional reader studies are indicated," Lehman and colleagues wrote. "Even larger reader studies will be needed to detect changes and improvements in reading accuracy."
They also recommended that educational programs on CAD be designed to improve both overall diagnostic accuracy and the efficiency of MRI interpretation.
