Incorporating computer-aided detection (CAD) results into the initial polyp search when reading virtual colonoscopy studies is more efficient than using CAD as a second reader, according to a multicenter study from London.
At the same time, sensitivity was maximized when CAD was used in its traditional role as a second reader, although the sensitivity difference between the two reading approaches didn't quite reach statistical significance, according to the researchers.
"The classic CAD implementation, 'second reader' CAD, is applied only after the observer has completed a full, unaided assessment," explained principal investigator Dr. Stuart Taylor and colleagues from London's University College Hospital and seven other institutions in Europe and the U.S. (Radiology, February 2008, Vol. 246:2, pp. 463-471).
"In this paradigm, CAD acts as a safety net for abnormalities missed by the unassisted reader. However, such implementation increases interpretation time," they wrote. "A potentially more time-efficient paradigm is 'concurrent read' CAD, which applies CAD at the start of the assessment. Although an intuitively attractive proposition, there is some evidence from breast and pulmonary literature that concurrent application of CAD reduces observer vigilance, reducing sensitivity. It is unclear whether this phenomenon applies to CT colonography" (CTC, or virtual colonoscopy [VC]).
The prospective study sought to compare the diagnostic performance and time efficiency of both second- and concurrent-read CAD paradigms for retrospectively obtained virtual colonoscopy datasets, using colonoscopy results from the same cases as a reference standard.
In the study, 10 radiologists (including five who were highly experienced at VC interpretation and five who were formally trained but less experienced) each read a set of 24 VC datasets containing a total of 69 colonoscopy-confirmed polyps (28 were 1-5 mm, 41 were ≥ 6 mm, and five cases were normal), using a workstation (Vitrea, Vital Images, Minnetonka, MN) integrated with CAD software (ColonCAR 1.2, Medicsight, London).
All 25 patients had undergone a full cathartic bowel prep using sodium picosulphate (n = 19) or sodium phosphate solution (n = 6). The patients were scanned in a four-detector-row CT scanner using 1.25- to 2.5-mm collimation, 120 kVp, and 100-200 mAs. The studies were considered clinically adequate.
Each reader examined the 25 cases twice -- once with concurrent CAD and once with CAD as a second reader. The two reading paradigms were separated by six weeks to minimize recall bias. In addition, readers were encouraged to examine the data in small blocks rather than in a single sitting, mirroring clinical practice, and they weren't told that the second reading consisted of the same datasets (cases allocated randomly) as in the first reading.
Readers were free to use the full functionality of the workstation for both concurrent and second-reader interpretation. CAD marks could be turned on and off during CAD-assisted reads, Taylor and colleagues noted.
Performance of CAD approaches in polyps 6 mm or larger, compared to unassisted reading:
|
The odds of improved detection of polyps 6 mm or larger with CAD compared to unassisted reading were 1.5 and 1.3 using CAD as a second and concurrent reader, respectively. In addition, the researchers completed the concurrent reads 2.9 minutes faster than the second reads.
"Observer experience influenced polyp detection, with the odds of detection by experienced readers 2.5 (95% confidence limits: 1.1, 5.6) times greater than those of less experienced readers," they wrote. "When small polyps (1-5 mm) were included in the model, the effect of reading method on polyp detection approached, but did not reach, significance (p = 0.06).
However, there was evidence of improved sensitivity using the second-read paradigm, inasmuch as seven of 10 readers found more polyps compared with the concurrent reading.
"Compared with the CAD unassisted read, the odds of polyp (any size) detection by using CAD as a second reader (OR, 1.4 [95% confidence limits: 1.1, 1.8]) were again greater than were those using CAD concurrently (OR, 1.04 [95% confidence limits: 0.78, 1.39])," the group wrote.
Compared to second-read CAD, the odds of detecting lesions, excluding and including polyps 1-5 mm, using CAD concurrently were 0.87 (95% CI: 0.59, 1.3) and 0.76 (95% CI: 0.57, 1.01), respectively.
Assuming that individual reader threshold levels for reporting or ignoring diminutive polyps (1-5 mm) remained constant throughout the study, the second-read CAD paradigm held an advantage for the smallest lesions despite its low CAD detection rate.
The authors speculated on the reasons why some polyps found by readers without CAD were ignored during a current read with CAD.
"It seems reasonable to expect readers to rush analysis of the unprompted colon image during the concurrent read, instead of focusing on each CAD mark," Taylor and colleagues wrote. "We did find some evidence supporting this; CAD did not prompt 32% of the polyps missed by at least one reader that were correctly identified on the unassisted read. By definition, 68% of polyps missed were correctly marked by CAD."
As for why readers ignored more CAD-prompted polyps when reading concurrently, the authors noted that mammography researchers Zheng et al showed that reduced sensitivity for breast cancer detection with concurrent CAD reading was worst when the CAD false-positive rate was highest, suggesting that too many CAD prompts can lead to readers placing less weight on correct prompts. The results of the present study suggest that the same phenomenon is occurring with VC CAD, the group wrote.
"The concurrent read was significantly more time-efficient than the second read, which has implications for work flow," they concluded. "Arguably, time efficiency far outweighs the reduced sensitivity for diminutive polyps."
By Eric Barnes
AuntMinnie.com staff writer
February 13, 2008
Related Reading
Radiologists more likely to reject certain true-positive CAD findings, January 22, 2008
Mammo CAD results show reproducibility in serial exams, January 10, 2008
'Patch view' may improve efficiency of VC CAD viewing, November 29, 2007
VC CAD beats human readers in multicenter trial data, November 19, 2007
Polyp ranking scheme boosts CAD efficiency, October 1, 2007
Copyright © 2008 AuntMinnie.com
