The best virtual colonoscopy readers don't rush through their cases, and they don't try to squeeze more than one image on the monitor at a time. But sticking to their preferred reading method doesn't help them find more polyps, according to a new analysis of data from the National CT Colonography Trial (ACRIN 6664).
Several studies have looked at different virtual colonoscopy (also known as CT colonography, or CTC) techniques, including 2D versus 3D interpretation, endoscopic 3D versus virtual dissection and other novel viewing methods, and, of course, the effects of using different workstations. But although a number of studies have asked patients what kind of exam they prefer (they're not sure), few have asked VC readers -- particularly the highest-performing readers -- how they prefer to look at VC images.
A new analysis of the multicenter National CT Colonography Trial published last fall is providing a trove of information about how readers like to read -- and how their reading techniques affect their performance. By analyzing the behavior of 15 readers who participated in the trial, Dr. Amy Hara from the Mayo Clinic in Scottsdale, AZ, and colleagues aimed to find out what made some readers excel.
The key question, said Hara in a telephone interview with AuntMinnie.com, was "What are good readers doing and how is it different from people who are average readers?" The study goals actually changed soon after the work began, she added. The original aim was to determine if specific interpretive preferences affect the performance of all readers at CTC interpretation.
"There weren't a lot of important differences," Hara said. "So we looked at it a different way, dividing readers into two groups: those who had at least 85% sensitivity and specificity, and those who had one or both below that."
The reader cohort included 15 radiologists at 15 institutions, both private practice and academic centers. Experience levels varied, but all readers had been trained and performance-tested in the detection of colorectal lesions 1 cm or larger with at least 90% sensitivity at VC. The ACRIN 6664 trial examined a total of 2,531 screening, average-risk patients with virtual colonoscopy, followed by same-day optical colonoscopy at the 15 centers.
For the study, Hara and her colleague Dr. C. Daniel Johnson from the Mayo Clinic, along with Meridith Blevins and Mei-Hsiu Chen, Ph.D., from the ACRIN Biostatistics Center at Brown University in Providence, RI, divided the readers into two groups based on their performance for detecting polyps 10 mm or larger in the trial, Hara said.
There were eight so-called super readers (group 1) who performed with both sensitivity and specificity of at least 85%. There were also six average readers, so defined for having fallen below 85% in sensitivity, specificity, or both.
All readers were asked to complete a survey that asked about their interpretation preferences during the study. Most questions focused on 2D interpretation, inasmuch as all of the 3D images were interpreted in the same method -- bidirectional endoluminal viewing. The survey responses were then compared between the super readers and the average readers.
Readers split on preferred method
Both groups of readers were asked the following questions:
- In clinical practice, do you prefer primary 2D or 3D, or do you have no preference?
- Do you use the entire monitor to display an axial image, or do you display several images on the monitor at once?
- Are 2D axial images evaluated with a large field-of-view (less magnification) or a small field-of-view (greater magnification)?
- For 2D supine and prone image interpretation, do you scroll in one direction only? Both antegrade and retrograde? Do you use multiplanar reformatted images?
The study team also determined for each reader whether interpretation of prone and supine images was synchronized, along with their interpretation times and previous experience reading CTC studies.
Those who preferred primary 2D or 3D were fairly evenly split. At the beginning of the trial, five of seven average readers preferred primary 2D reading, while two had no preference. By the end of the trial, five of these readers still preferred primary 2D but two had switched to primary 3D. Among the eight super readers, three preferred 2D while one preferred 3D at the start of the trial. Two had no preference. By the end of the trial, two still preferred 2D but six no longer had a preference.
"Most readers liked 2D by the end of the study. It was really the same trend for both groups of readers," Hara said, although "a couple had switched to primary 3D."
Performance unaffected by preference
More surprisingly, reader performance was not affected by the reader's preferred primary reading method. For readers reporting a primary 2D preference, performance was similar whether the reader used a primary 2D (sensitivity/specificity = 89%/87%) or primary 3D (sensitivity/specificity = 83%/84%) method. Similarly, no differences using a primary 2D or 3D method were identified when readers preferred primary 3D or had no reading preference, according to Hara.
|
|||||||||||||||
| Reader performance was not affected by the preferred reading method. In this table, super reader preference did not affect performance for polyps ≥ 10 mm in size. In the following table, average reader preference did not affect performance for polyps ≥ 6 mm in size. All table data courtesy of Dr. Amy Hara. |
|
"Some people are better at 2D than 3D, but it's pretty clear from the results that not everybody is going to be better at one or the other -- it's just not coming out that way," Hara said. "And the majority of super readers really had no preference."
Habits of highly effective readers
The study also recorded the habits of highly effective readers. In 2D interpretation, for example, all super readers scrolled through the axial images in both directions for the supine data, and 88% did the same for the prone data.
Three-fourths of the super readers filled the monitor screen with a single axial image, while only 43% of the average readers did so. The rest tended to examine several images on the monitor at once, effectively making each image smaller, Hara told AuntMinnie.com.
|
The super readers also took longer to read each case. Super readers required a mean 20 minutes to interpret a case in 2D and 27 minutes in 3D, versus 19 and 24 minutes, respectively, for average readers.
|
||||||||||||||||
| Super readers had slightly longer interpretation times compared to average readers. |
As a practical matter, the best reading method appears to depend more on the case than on the reader, Hara said. "If you have a great prep, 3D is really easy; if you have a poor prep, where there's lots of stuff in there, then 2D is sometimes easier." For her own cases, Hara said she tends to go back and forth between methods. "I'll start an exam one way or the other, but I always look at an exam with both," she noted.
Summing things up, the results for 2D image review showed better sensitivity with no significant decrease in specificity when axial images were the only images displayed on the monitor (sensitivity = 94%), versus when axial images were simultaneously displayed with 3D or multiplanar reformatted images (sensitivity = 74%, p = 0.04). It appears to be important to fill the screen with a single image, and the effects of reader fatigue may also be magnified when readers are looking at very small images, Hara said.
Second, the best readers scrolled through both the prone and supine axial images in both directions, Hara said. For all readers, however, performance was not improved when prone images were evaluated in both directions versus just one direction only, when done after evaluating supine images in both directions (i.e., rectum to cecum and reverse).
Things that didn't appear to matter in sensitivity and specificity included the use of multiplanar reformatted images. In addition, synchronizing the prone and supine images appeared to have little effect on performance.
As for the number of cases read, "the groups were equally divided in terms of experience," she said. "Super readers were not more experienced on average."
|
||||||||||||
| Super readers and average readers were fairly evenly divided in terms of the number of CTC cases they had read. |
Probably the most surprising finding for the study team was reader preference's lack of effect on performance. "What you liked to do didn't really have an impact, which I thought was interesting," Hara said. "I think that most of us would think that what we like to do would be what we do best."
Hara cautioned that the results aren't necessarily transferable to all readers and all studies. "The results depend so much on the group of readers you get," she said. However, the methods shown to make a clear difference in performance should be considered for use in CTC training programs, Hara said.
By Eric Barnes
AuntMinnie.com staff writer
February 16, 2009
Related Reading
ACRIN VC trial shows only small differences in bowel preps, February 9, 2009
Virtual colonoscopy beginners do better with 3D data, February 5, 2009
Need for VC colon screening trumps physician turf battles, January 20, 2009
Same-day VC prep works, but some preps are better than others, December 16, 2008
ACRIN virtual colonoscopy training tied to performance, October 27, 2008
Copyright © 2009 AuntMinnie.com
