Computer-aided detection (CAD) software can boost the performance of even expert radiologists in finding challenging polyps on virtual colonoscopy studies, say researchers from the University of Chicago.
A research team led by Kenji Suzuki, PhD, found that the use of CAD software as a second reader delivered a statistically significant improvement in the sensitivity of three expert radiologists and one adequately trained radiologist in detecting difficult polyps on VC (also known as CT colonography or CTC).
Although previous research has shown a high standalone performance for CAD systems in detecting clinically significant polyps at CTC, it doesn't necessarily follow that CAD will improve the performance of radiologists. Radiologists are able to detect polyps that aren't difficult without computer aid, and they may also reject correct computer detections, according to Suzuki.
As a result, observer performance studies with radiologists are needed to measure the actual usefulness of CAD, Suzuki said.
"Although some researchers conducted such observer performance studies recently, none of them used 'difficult' polyps in their studies. And expert radiologists did not participate in most of the studies, i.e., computer-aided detection systems [were] proved to be useful for novice or nonexpert readers, but not for expert readers," Suzuki told AuntMinnie.com. "Therefore, [the] actual usefulness of a computer-aided detection system on the performance of expert radiologists in the detection of difficult polyps was unknown."
To measure the technology's effect on expert radiologists in detecting difficult polyps, the researchers sought to evaluate the use of CAD based on massive-training artificial neural networks (MTANNs). He presented the team's findings during a scientific session at the 2010 RSNA meeting in Chicago.
The institution's CAD algorithm uses a fairly standard approach, beginning with colon segmentation, followed by detection of polyp candidates based on shape analysis and then analysis of morphologic and texture features, according to Suzuki. The final step, however, features the use of MTANNs for reducing false positives.
For the purposes of the study, "difficult" polyps were defined as polyps that were initially missed by radiologists in the multicenter clinical trial led by Donald Rockey, MD, (Lancet, January 22, 2005, Vol. 365:9456, pp. 305-311) or true-positive cases rated as including a "difficult" polyp in a retrospective review of the cases by an expert radiologist.
That study included 459 polyp-free patients and 155 patients with 234 polyps 6 mm or larger. Of these 234 polyps, 120 were true-positive polyps on CTC and 114 were false negatives on CTC.
The University of Chicago researchers created their study cohort by randomly selecting 20 patients with at least one difficult polyp or a false-negative polyp on CTC. Of these 20, 13 were false-negative patients with 14 false-negative polyps and seven were true-positive patients with seven "difficult" polyps and two "easy" polyps.
Ten negative cases also were randomly selected from the trial.
Four board-certified abdominal radiologists participated in the study, three of whom were CTC experts who had read more than 700 cases, Suzuki said. The fourth radiologist had read more than 50 positive cases in standardized training.
In a training session prior to the study, the readers went through six training cases. The readers were then asked to read the study cases, first without and then with CAD, to indicate the location of polyps and provide their confidence level regarding the presence of polyps (using a continuous rating scale, where 0 = definitely negative and 1 = definitely positive).
Readers could use 3D endoluminal and/or 2D multiplanar views for polyp detection and problem solving. They were blinded to the prevalence of polyps but were informed of the general performance of the MTANN CAD software, according to Suzuki.
The researchers then calculated mean sensitivity per polyp, positive predictive value by patient, and figure of merit (i.e., area under the curve) from jackknife alternative free-response receiver operator characteristics (JAFROC) analysis.
Mean reader performance, with and without CAD
|
"It was proved that a computer-aided detection system can improve expert radiologists' sensitivity in detection of difficult polyps in CT colonography," Suzuki told AuntMinnie.com.
The increases for sensitivity and figure of merit were statistically significant (p = 0.037 and p = 0.006, respectively), but they were not significant for positive predictive value (p = 0.098).
 |
| A polyp (7-mm sessile adenoma in the rectum; false negative on CTC in the original trial) initially missed by two out of four radiologists in the study, but detected with CAD. Images courtesy of Kenji Suzuki,PhD. |
In other findings, average per-polyp sensitivity increased from 47% to 60% for polyps 6 to 9 mm in size. For polyps 10 mm and larger, mean sensitivity increased from 66% to 69%.
Suzuki acknowledged a number of limitations in the study, including the lack of statistical significance for the difference in positive predictive value due to the small number of readers. In addition, the difficult cases were determined by a single expert reader, he said.
Nonetheless, the researchers concluded that the CAD system with MTANN technology improved the diagnostic performance of radiologists -- including expert readers -- for detecting difficult polyps.
"CAD had the greatest impact in the detection of difficult polyps," Suzuki said.
By Erik L. Ridley
AuntMinnie.com staff writer
February 14, 2011
