Computer-aided detection (CAD) is known to boost radiologist performance in the detection of lung nodules, and radiologists certainly need the help. But despite a decade of fine-tuning and some powerful new capabilities, CAD systems haven't found a place in daily practice at most imaging facilities
Yet with widespread lung cancer screening seemingly just around the corner, radiologists are finding more reasons to hope lung CAD will be drafted to join their teams sooner rather than later. The newest algorithms do more than detect solid nodules, they evaluate subsolid lesions and ground-glass opacities. At the same time, research is pointing the way to the best and most efficient ways for radiologists to interact with the technology.
"There are certain areas in CT where computer vision has been applied for some time -- and I think it could be argued that there's probably no more urgent need than the assessment of lung nodules," said Dr. Geoffrey Rubin, a professor of radiology and bioengineering at the Duke Clinical Research Institute at Duke University in Durham, NC.
Success: A long time coming?
Why lung CAD? Because lung nodule detection is extremely heterogeneous: Nodule detection and comparison is hard, time-consuming work; multiple studies have found radiologists' nodule detection skills all over the map; and CAD always improves detection efforts, Rubin said in a June talk at the 2014 International Symposium on Multidetector-Row CT in San Francisco.
Success has been a long time coming. A decade ago, Rubin and colleagues at Stanford University assessed reader performance in detecting 190 nodules 3 mm and larger without the aid of CAD. In the end, no reader found more than 59% of the nodules on his own (Radiology, January 2005, Vol. 234:1, pp. 274-283).
More recently, Rubin and colleagues digitally embedded 157 nodules 5 mm and larger into 41 CT datasets. Thirteen experienced radiologists read the data without CAD and found not quite half of them -- 49% in the aggregate.
A study of 157 digitally embedded 5-mm nodules showed that radiologists performed poorly in the task of finding nodules without CAD. Overall, 13 readers found only 49% of the clinically relevant lesions. Images courtesy of Dr. Geoffrey Rubin."So we really do need help," and studies are clear that CAD makes a big difference, he said.
Unveiling CAD's secrets
In the past decade, CAD has learned some new tricks. It's increasingly being applied to chest radiography, for one thing, and in the search for subsolid and ground-glass nodules, for another -- an important advance considering that ground-glass opacities are likelier to harbor malignancies than other nodule types. Also on the front burner is research aimed at determining how lung CAD works in the environment of low-dose CT lung cancer screening that is taking shape, Rubin said.
"One of the more intriguing elements [of current research] is looking at the role CAD should play," Rubin said. "We have traditionally considered CAD from a very conservative perspective as a second reader" -- i.e., when the radiologist reads the scan unaided then turns on CAD and either accepts or rejects the markings.
But now some studies are using CAD as a concurrent reader, or as a first reader, when the radiologist looks only at what CAD has already identified as a finding. "In each of these are implications for diagnostic performance as well as efficiency," Rubin said.
Detect thin, read thick
A study published last year by Godoy and colleagues (American Journal of Roentgenology, January 2013, Vol. 200:1, pp. 74-83) was particularly informative regarding the optimum interaction of CAD and reader. Using a work-in-progress algorithm, investigators looked at CAD performance alone, and CAD/radiologist performance in several combinations, including:
- CAD analyzing thick sections and readers looking at thin sections
- CAD analyzing thin sections and readers looking at thick sections
- CAD analyzing thin sections and readers looking at thin sections.
The study included 155 nodules: 74 solid, 22 part solid, and 59 ground-glass opacities, with a 5.5-mm average diameter. Four readers looked at various combinations of thick (5-mm) and thin (1-mm) slices.
Dr. Geoffrey Rubin from Duke University Medical Center.All reading paradigms that involved CAD reading thin sections alone were better (71% to 95%) than the reader working without CAD (sensitivities 57% to 81%), though, predictably, CAD working alone had a higher false-positive rate of about three per case.
The reading paradigm with the highest sensitivity (82% to 97% depending on nodule type) occurred in CAD when both the reader and CAD analyzed thin sections. The worst performance (sensitivities of 40% to 77%) occurred when the reader was asked to assess marks that CAD had detected on thick sections. This is because it's harder for the reader to discern whether CAD markings from thick sections are true-positive or not, Rubin said.
But even though the CAD thin/reader thin paradigm delivered the highest sensitivity, CAD thin/reader thick (sensitivities 77% to 94%) was actually the optimal combination: It nearly equaled the sensitivities of CAD thin/reader thin (82% to 97%) but produced less uncertainty for the reader, leading to fewer false positives.
Consequently, the authors reported that false positives rose from 0.64 to 0.90 per case (compared with no CAD) when CAD and the reader both examined thin sections (p < 0.001), but false positives didn't increase when CAD examined thin sections and the reader looked at thick.
Problems associated with reading CAD marks on thin sections included greater uncertainty, and longer reading times have also been documented in previous studies.
"When the reader is presented with thick sections, they have many fewer sections to look at. It's a more efficient paradigm, and the reduction in sensitivity isn't that great," Rubin said. Therefore, the most effective reading paradigm was CAD thin/reader thick, "allowing the radiologist to comfortably and confidently read thick sections, and then look at CAD's premarkings and evaluate the thin sections to validate that and perform at a high level -- almost as high as if the radiologist read exclusively thin sections."
Concurrent reads, wide variation
Almost all lung CAD studies have begun with the reader looking at the dataset unaided, followed by CAD processing the CT dataset, and the reader accepting or dismissing the CAD markings. But a study by Matsumoto and colleagues in the European Journal of Radiology (EJR, August 2013, Vol. 82:8, pp. 1332-1337) found that using CAD concurrently with reader review was twice as fast (132 seconds per read for concurrent versus 210 seconds traditional) and nearly as accurate (70% sensitivity for concurrent reading versus 72% for CAD as a second reader). In light of this, is CAD as a second reader a waste of time?
Also from EJR comes evidence that current commercial CAD schemes perform poorly -- or at least turn up vastly different findings (December 2013, Vol. 82:12 pp. e873-878). Christe and colleagues loaded up a lung phantom with nodules and compared three commercially available CAD schemes at eight low-dose CT levels. The reader plus CAD was the best reading paradigm, but the algorithms showed huge variability in the findings, Rubin said. Newly developed systems are likely to do better.
"As independent readers, we are not very adept at performing the detailed search required to find nodules," Rubin said in summary. "Lung CAD results are extending to subsolid and ground-glass opacities, but the best systems at this point are not commercially available. Time savings as well as diagnostic improvements are apparent, and the main question is when these tools will become available."
