Computer-aided detection (CAD) technology can be a valuable tool for improving the detection of cerebral aneurysms, but researchers from the University of Tokyo found that a lack of reader confidence in CAD findings can impede its effectiveness when used for MR angiography (MRA) screening.
Used as part of a routine reading protocol of 3D MRA in an annual whole-body health screening service at their institution, CAD was able to detect aneurysms that had been missed by two radiologists in their initial interpretations. Unfortunately, however, the readers disregarded many of the true-positive CAD marks in the 14-month study.
"Although our CAD system presented a comparable number of lesions to human observers, radiologists were likely to ignore CAD results and overlook small aneurysms," said Dr. Soichiro Miki. "Improvements in [lesion] candidate display effectiveness may yield higher sensitivity."
Miki presented the findings during a scientific session at the RSNA 2011 meeting in Chicago.
Whole-body health screening
At the University of Tokyo, 3D MRA exams are routinely performed as part of an annual whole-body health screening service. An internally developed CAD application for cerebral aneurysm is used within their routine reading environment, Miki said.
The CAD application runs off a Web-based CAD platform called Clinical Infrastructure for Radiologic Computation of Unified Solutions (CIRCUS) CS. This platform, which was developed by the university, provides a framework for common CAD-related tasks such as DICOM connections, job queue management, and results presentation, according to Miki.
The MRA-CAD application for aneurysms is a plug-in on CIRCUS CS and is configured to display three lesion candidates for each subject on its Web-based user interface.
To evaluate the performance of the CAD application and the level of confidence radiologists have in CAD results in routine reading, the researchers studied 616 asymptomatic adults who received an initial time-of-flight MRA study on a 3-tesla MR scanner between July 2010 and September 2011.
Two radiologists with at least four years of experience then independently interpreted the images. They were able to view both volume-rendered and axial slices with no limitations on reading time. The diagnostic criterion for an aneurysm was a protrusion of 2 mm or more.
In the institution's routine reading protocol in the health screening service, two radiologists interpret images without CAD and provide an initial diagnosis. They then independently review the CAD results and press a "Missed TP" button on the application if their diagnosis has changed, Miki said.
A technologist also reads the MRA without CAD and contributes his or her opinion. In the final step in the process, the two radiologists discuss the case and render the final diagnosis by consensus.
Of the 616 subjects included in the study, 51 (8.3%) were determined to have at least one aneurysm, as determined by the final diagnosis.
Using the final diagnosis as the gold standard in the study, the two radiologists had 60.8% sensitivity and 99.1% specificity for their initial diagnosis, and 66.7% sensitivity and 98.9% specificity after viewing the CAD results. However, the CAD application displayed the actual aneurysms in 70.8% of cases.
Of the 1,156 total cases in which the radiologist had an initial "negative" diagnosis, eight were changed to positive based on the CAD results. Seven of these were true-positive aneurysms and one was a false-positive result.
However, 33 aneurysms were missed by the radiologists even though they had been included in the CAD results. "This means [they] overlooked aneurysms even with the aid of CAD," Miki said. There was also low concurrence among readers.
Reader performance at MRA screening
|
In the four cases that received a final "positive" diagnosis after an initial "negative" reading by both readers, the actual lesion was pointed out only by CAD or the radiologic technologist.
In other findings, both the readers and CAD had difficulty detecting small lesions, according to the group.
Reader and CAD performance by aneurysm diameter
|
The CAD results were less likely to affect human decisions than they had expected, Miki said. "The reason for this may be that our CAD system presented too many false positives, distracting the reader's attention," he noted. "Another reason may be that some lesions are difficult to notice in axial slices."
While the performance of a CAD system may typically be measured by the free-response receiver operator characteristics (FROC) curve, the real usability of CAD may be largely based on how effectively the results are displayed, Miki said.
