When it comes to reading virtual colonoscopy studies, a radiologic technologist is no match for a trained radiologist. But two technologists are another matter entirely: They were equivalent to, and maybe slightly better than, a single radiologist, according to a new study in Radiology.
In a Dutch study that compared experienced radiologists with technologists trained on hundreds of virtual colonoscopy studies, the technologist pairs found most of the true positives identified by the experienced radiologists, while generating far fewer false positives than a single radiologist (Radiology, September 2012, Vol. 264:3, pp. 771-778).
"We found that a reading strategy of two technologists ... resulted in a comparable sensitivity for advanced neoplasia but with far fewer [false-positive] referrals compared with reading by one radiologist," wrote Dr. Margriet de Haan, PhD, and colleagues from Academic Medical Center in Amsterdam, along with others from Erasmus University Medical Center in Rotterdam.
Shortage of trained radiologists?
As growing demand for VC (also known as CT colonography or CTC) interpretation clashes with a shortage of CTC-trained radiologists, trained technologists could be part of the solution, they wrote.
While acknowledging the medicolegal issues involved in handing more responsibility to technologists, the authors noted that technologists have performed well in interpreting CTC images, as demonstrated in previous studies. Most have shown comparable diagnostic accuracy between technologists and radiologists for intracolonic colorectal lesions, they wrote.
Technologists have performed well enough, in fact, that they could potentially serve as primary readers in the screening of large cohorts, while the search for extracolonic interrogations would likely still be limited to radiologists.
The goal of the current study was to compare the diagnostic yields of CTC-trained radiologists with those of trained technologists for detecting advanced neoplasia. The study included 982 participants (507 men, 475 women) who underwent virtual colonoscopy scans on a 64-detector-row scanner (Brilliance, Philips Healthcare) at 64 x 0.625-mm collimation, with 0.9-mm-thick reconstructions, 120-kV tube voltage, and 25 mAs. The scans followed a noncathartic bowel preparation with iodinated oral contrast.
Each dataset was evaluated by one of three experienced radiologists using a primary 2D reading technique followed by secondary computer-aided detection (CAD) analysis. Polyps 10 mm and larger were referred for colonoscopy, with surveillance recommended for 6- to 9-mm lesions. Detection of advanced neoplasia was defined as a true-positive reading.
Each scan was also evaluated by two of four technologists trained in CTC, using a primary 2D technique followed by secondary 3D reading and CAD. Both radiologists and technologists decided cases by consensus in case of disagreement.
Three of the technologists had no experience with CTC before the trial, while a fourth had completed 50 cases. For the study, all four successfully completed a structured training program that included 175 training cases and 25 test cases verified by conventional colonoscopy. The three radiologists also were trained and experienced in CTC interpretation. Before participating in the study, all readers had examined a minimum of 200 colonoscopy-verified cases.
The technologists achieved 71% to 100% sensitivity in their test cases following training, de Haan and colleagues wrote. The investigators analyzed six separate reader scenarios: one radiologist, one technologist, one of each without consensus, two radiologists with consensus, two technologists without consensus, and a radiologist and two technologists without consensus.
Sensitivity and specificity of CTC couldn't be calculated because not all individuals underwent conventional colonoscopy, the authors explained. As an alternative, sensitivity and false-positive referrals were compared between different readers and reading strategies by calculating true-positive fractions and false-positive fractions. Detection of advanced neoplasia (classified by a pathologist) was defined as a true-positive finding. Relative true-positive and false-positive fractions were calculated along with 95% confidence intervals (CI).
Using this method, a relative true-positive fraction greater than 1 indicated that the alternative strategy (technologists) was more sensitive than the reference category (radiologist), while a relative false-positive fraction greater than 1 showed that the alternative strategy produced more false-positive referrals compared with the reference category, the authors noted. The aim was to find reading strategies with more true-positive and fewer false-positive referrals than the strategy of one radiologist.
A reading strategy of one radiologist plus one or two technologists produced significantly better sensitivity for advanced neoplasia and a comparable number of false-positive referrals when compared with the one-radiologist strategy, the study team wrote.
For example, compared to a radiologist, two technologists working together found a comparable number of advanced neoplasia (relative true-positive fraction = 0.92, 95% CI: 0.78-1.07). However, they had far fewer false-positive findings than the radiologist (relative false-positive fraction = 0.38, 95% CI: 0.21-0.67).
Double reading by two technologists led to comparable detection rates (5.6 and 5.9 detections, respectively, in 100 participants) as the radiologist, who had only slightly more detections (6.1 per 100 participants).
Finally, one radiologist and either one or two technologists produced significantly better sensitivity for advanced neoplasia (relative true-positive fraction = 1.10 and 1.13, respectively). The number of false-positive referrals was comparable to one radiologist.
Sixty (71%) of the 84 participants referred for colonoscopy by the radiologists had advanced neoplasia, compared with 55 (86%) of the 64 participants referred by two technologists, the authors wrote. The radiologists and technologists all detected all five colorectal cancers.
The mean reading time for the radiologists was 307 seconds ± 156 (range, 100-1,297 seconds) for 2D interpretation plus CAD. The technologists' mean reading time was 506 seconds ± 219 (range, 171-2,131 seconds).
The use of secondary computer-aided detection by the radiologists led to the identification of one additional screening subject with advanced neoplasia. The technologists' use of CAD did not increase the number of referrals.
Based on a reading strategy of one technologist or one radiologist and one technologist without consensus, both 3D and CAD readings yielded one additional participant with advanced neoplasia, de Haan and colleagues wrote. The use of secondary 3D reading and a third CAD reading by two technologists who were not allowed to diagnose in consensus yielded no additional referrals for colonoscopy.
"By using a cutoff value of at least 10 mm for referral to colonoscopy, we found that a CT colonography reading strategy of one technologist resulted in a significantly lower diagnostic yield for advanced neoplasia compared with that of the radiologist, whereas double reading by two technologists with or without consensus resulted in a comparable diagnostic yield," the authors wrote. "Reading strategies of one radiologist and one or two technologists (without consensus) would result in a higher sensitivity for advanced neoplasia compared with a reading strategy of one radiologist, with a comparable number of [false-positive] findings."
As for study limitations, the authors noted that colonoscopy confirmation of findings was not available, but that it didn't affect the main findings of the study, which was designed to evaluate detection differences among different reader groups. Results in participants referred for surveillance for three years may increase the number of advanced neoplasia when these findings become available, the authors wrote.
Reading by a single radiologist plus one or two technologists yielded more true-positive advanced neoplasia than a single radiologist alone, de Haan and colleagues concluded, and reading by two technologists produced comparable sensitivity to a radiologist with far fewer false-positive referrals.
This indicates that two technologists could serve as primary readers, with radiologists handling the search for extracolonic findings. However, "it will depend on screening legislation whether technologists could have the final liability for the intracolonic findings," they wrote.
