Virtual colonoscopy CAD systems continue to grow in complexity and performance, and it is only a matter of time, developers say, before a system is approved for commercial use. Yet the kinds of studies needed to underpin such regulatory approvals are really just getting started.
The first large multicenter trial of a VC CAD scheme, under way at 12 centers in the U.S. and Europe, is showing real promise, with high sensitivity and specificity for detecting colorectal polyps.
In another recent study, researchers in Japan used a smaller study cohort to ask some intriguing questions of their own. Their results showed that CAD improved the ability to detect colorectal polyps by everyone who used it, including radiologists, radiology residents, and endoscopists. In a third study, however, CAD missed some smaller polyps in a low-dose protocol. All of the studies described in this article were presented at the 2004 RSNA meeting.
Around the world with CAD
"The objective of the study was to evaluate the sensitivity and generalizability of a CAD algorithm obtained from multiple data sites," said Luca Bogoni, Ph.D., from Siemens Medical Solutions' CAD group in Malvern, PA. "The goal was to have CAD as an add-on to workflow."
Five institutions, including the New York University Medical Center in New York City, the Cleveland Clinic Foundation in Ohio, as well as sites in Indiana, Austria, and Belgium, participated in the initial study of 259 patients at above-average risk of colorectal polyps and cancer. Of these, data from 96 patients was used to "train" the CAD algorithm, which was then tested in the remaining 67 patients.
The retrospective study has since grown to 500 patients at 12 institutions, and the CAD results have continued to improve, Bogoni told AuntMinnie.com.
Various scanners and protocols reflecting the variety found in daily practice were used to create the datasets, Bogoni said. The scanners, all from Siemens Medical Solutions, ranged from the four-slice Volume Zoom to the 16-slice Sensation. The slice thickness ranged between 1.25 mm, 1.5 mm, and 2 mm, with reconstruction intervals ranging from 0.7 mm to 1.2 mm. Exposure ranged from 50-110 mAs, while kVp was fixed at 120. All the patients underwent purgative bowel cleansing and room-air insufflation before imaging, and nearly all underwent prone and supine imaging.
A few VC cases resulted from incomplete colonoscopies, and some from incomplete insufflation and residual fecal material, Bogoni said.
To gauge the results, so-called ground truth was established by means of virtual colonoscopy, followed the same day by conventional colonoscopy in all patients.
The CAD chose polyp candidates based on shape, texture, and CT-attenuation characteristics, Bogoni said. The features were extracted at the location of each polyp candidate, and finally, suggestion of polyp candidates was generated for the radiologist's review.
The results showed that the 96 training cases (188 volumes) contained 76 polyps, including 31 small (1-5 mm), 32 medium (6-9 mm), and 13 large (10-20 mm). The 67 test cases (133 volumes) contained 53 polyps, including 21 small, 19 medium, and 13 large polyps.
 |
| Above, Siemens investigational VC CAD system. A marker ("c3a") is visible in the endoluminal view (lower right) representing a lesion proposed by the CAD system for physician review. The "c" indicates that it was automatically located. The other two markers (1a and 2a), also visible in the global view at lower left, were placed by the physician during the first read. The marker is highlighted in yellow because it is the one being reviewed. Images courtesy of Dr. Luca Bogoni. |
For the 96 training cases, sensitivity for detection vis-a-vis ground truth was 65% for small polyps, 94% for medium-sized polyps, and 85% for large polyps. In the 67 test patients, there were 53 polyps, including 21 small, 19 medium, and 13 large.
Like most radiologists, the CAD system improved after training. Sensitivity in the test cohort was 62% for the small polyps, 79% for the medium polyps, and 92% for large polyps. There was an average of 4.3 and a median of 4 false-positives in each test-cohort dataset.
"I think the take-home message is that the sensitivity provided by this system is on track with clinical sensitivity, and some of the false-positives we had were easily dismissed as the iliocecal valve, extracolonic findings, and such, so we're on the right track," Bogoni said. "We're looking to increase sensitivity, which indicates about 90% sensitivity for polyps over 6 mm, with about four false-positives per case or maybe a little bit less. This study was completed in March, and we've now covered up to 500 cases from 12 clinical sites."
Results for the additional 500 patients have not yet been finalized, Bogoni told AuntMinnie.com this week. "However, some improvements in the algorithms indicate that the overall sensitivity has improved substantially, and will be above 90% with few false-positives," he said.
The published results will demonstrate that CAD, used as a second reader, can substantially improve the sensitivity of readers.
Aiding reader performance in Japan
In another RSNA presentation, Dr. Masanori Imuta and colleagues from Kumamoto University in Honjyo, Japan, along with Dr. Hiroyuki Yoshida from the University of Chicago in Illinois, sought to determine whether CAD could improve VC detection in more than one type of reader. (Yoshida is now an associate professor of radiology at Harvard Medical School in Cambridge, MA, and a radiologist at Massachusetts General Hospital in Boston.)
The goal was to "evaluated whether CAD improves the performance of radiologists and endoscopists in the detection of colorectal lesions on CT colonography (VC)," Imuta said.
The group evaluated 50 cases, including (in the first 40) 35 with polyps larger than 5 mm, which were selected and examined retrospectively from patients who underwent standard purgative cleansing, bowel insufflation, and prone and supine CT imaging. All the findings were confirmed by conventional colonoscopy as the gold standard, he said.
A previously reported CAD scheme, set for a by-polyp sensitivity of 90% detection rate with two false-positives per case, was used to aid the 15 readers, who included five board-certified radiologists, five radiology residents, and five endoscopists. They read the cases at a CAD workstation using 2D primary interpretation with 3D endoluminal views for problem solving -- first without, then with CAD results.
They rated their confidence in the diagnosis, and when the CAD results were unveiled, were invited to change their confidence levels. The group performed an alternative free-response receiver operating characteristic (AFROC) analysis on the onfidence levels, and the area under the AFROC curve was used to compare the readers' performance with and without CAD.
Az increased for all 15 readers with the use of CAD. The average Az value for all reader groups was 0.51 with CAD and 0.72 without CAD, and the difference was statistically significant (p < 0.001), Imuta said. The differences in Az values was seen among reader types did not reach statistical significance, but trends could be seen among the reader groups in the small study. As a group, the endoscopists achieved the highest Az value (0.78). The board-certified radiologists scored 0.73, while residents chalked up an Az score of 0.65.
"The endoscopists benefitted the least from the use of CAD," Imuta said. Their improvement was 0.17, whereas board-certified radiologists and residents' improvement was 0.26 and 0.19, respectively.Smaller polyps missed in low-dose CAD
A second study by Imuta's group tested the same CAD system in low-dose studies. Twenty-one patients were scanned twice in prone and twice in supine, at 100-120 kVp and mAs set at 100 mAs and 20 mAs, respectively. The four-row scanner was set at 2.5-mm collimation, pitch 6, and conventional colonoscopy plus imaging revealed 25 polyps ranging from 5-30 mm in size (mean 13.5 mm). Three experienced radiologists reviewed the studies with the aid of CAD.
 |
| A 10-mm polyp at the splenic flexure. The polyp is visible in both low-dose (upper left) and standard-dose (upper right) CTC images. CAD detected the polyp in both doses (lower left and right). Images courtesy of Dr. Masanori Imuta and Dr. Hiro Yoshida. |
CAD's by-polyp sensitivity for polyps of all sizes was 83.3% for standard dose studies and 77.8% for ultra-low-dose exams, with an average of four false-positives per patient. With CAD, the three readers’ average sensitivities for standard- and low-dose studies were 100% and 88.9% (≥ 10 mm), and 78.8% and 60.1% (< 10 mm), respectively, with an average 2.3 false-positives per study.
 |
| A 6-mm polyp is visible in standard-dose CTC (left) but invisible in low-dose CTC (right). Consequently, the polyp was not detected by radiologists or CAD in low-dose CTC. Images courtesy of Dr. Masanori Imuta and Dr. Hiro Yoshida. |
No statistically significant difference in the detection rate was seen between the standard- and low-dose protocols, however CAD with low-dose data performed poorly in detecting smaller lesions, Imuta said.
"The CAD scheme has the potential to detect colorectal lesions in patients with high sensitivity and a low false-positive rate, even on ultra-low-dose CTC," Imuta said. "However, sensitivity for detecting polyps 5-10-mm in size at 20 mAs was significantly lower (p < 0.05) than that at 100 mAs," the group wrote in an abstract.
By Eric Barnes
AuntMinnie.com staff writer
March 4, 2005
Related Reading
VC CAD system finds polyps in opacified fluid, January 3, 2005
Low-prep VC study finds CAD can be fooled, December 23, 2004
Mass appeal: VC CAD doesn't stop at polyps, October 10, 2003
Knowledge-guided segmentation improves polyp detection, October 18, 2002
Copyright © 2005 AuntMinnie.com
