CAD offers value in detecting lung nodules with CT

Dec 16, 2007

Computer-aided detection (CAD) technology offers a relatively high detection rate for detecting lung nodules on CT scans, according to two presentations at last month's RSNA 2007 meeting in Chicago.

In the first study, researchers from the University of Chicago in Illinois sought to analyze the performance levels of its CAD scheme for detecting lung nodules of various sizes and patterns on thin-slice CT studies.

The institution's CAD scheme consists of lung segmentation, selective nodule enhancement, initial nodule detection, accurate nodule segmentation, and feature extraction and analysis techniques. The selective nodule enhancement filter provides enhancement of nodules and suppression of other normal anatomic structures, such as blood vessels that are the main source of false positives, according to presenter Qiang Li, Ph.D.

The researchers utilized a database consisting of 117 thin-slice CT scans with 153 nodules, including 91 nodules obtained from 85 scans at Shinshu University in Nagano, Japan. The remaining 32 scans and 68 nodules were contributed by the University of Chicago.

Of the 153 nodules, 68 (44.4%) were small, 52 (34%) were medium-sized, and 33 (21.6%) were large nodules. There were 101 (66%) solid nodules and 52 (34%) nodules with ground-glass opacity (GGO) in the combined database.

A case-based, fourfold cross-validation method was employed to evaluate the performance levels of the CAD scheme. The researchers repeated the cross-validation testing method 10 times, and determined the average performance levels for all nodules.

With a setting of 6.5 false positives per scan, the CAD scheme yielded overall sensitivity of 87%, including sensitivity of 74% for small nodules, 98% for medium-sized nodules, 94% for large nodules, 85% for solid nodules, and 90% for GGO nodules.

With a setting of 2.7 false positives per scan, the scheme provided overall sensitivity of 82%, including sensitivity of 68% for small nodules, 94% for medium-sized nodules, 91% for large nodules, 78% for solid nodules, and 89% for GGO nodules. A setting of 1.5 false positives per scan led to overall sensitivity of 77%, including sensitivity of 63% for small nodules, 90% for medium-sized nodules, 89% for large nodules, 71% for solid nodules, and 89% for GGO nodules.

The detection scheme found 82% of the nodules with 2.7 false positives per scan, Li concluded.

"Our detection scheme can detect many nodules with GGO," Li said. "However, it is difficult for our CAD scheme to detect very small nodules between 4 and 7 mm."

Sensitivity gains

In another study presented at the meeting, a research team from the University of Michigan (UM) in Ann Arbor found that CAD improved the performance of radiologists in detecting lung nodules, particularly smaller ones.

Average figures of merit (FOM) by jackknife free-response receiver operating characteristic (JAFROC) analysis improved at all nodules sizes using CAD, according to presenter Berkman Sahiner, Ph.D. "The improvement was statistically significant with 3- and 4-mm thresholds, and approaches significance at a 5-mm threshold," he said.

To investigate the effect of the CAD system on the performance of thoracic radiologists in detecting pulmonary nodules on CT exams, the UM research team used a set of 85 CT scans, including 49 from the Lung Imaging Database Consortium (LIDC) and 36 from the University of Michigan. Six of the nodules proved to be malignant after biopsy.

To determine the reference standard of truth, the UM database was read by two expert cardiothoracic radiologists, while the LIDC database was read by four expert cardiothoracic radiologists, Sahiner said. A "true" location was defined as a region marked by at least one radiologist as containing a nodule larger than 3 mm.

Overall, 118 reference nodules 3 mm or larger were found in 52 exams; masses 3 cm or larger were excluded. Average nodule size was 6.8 mm.

After the CAD system was trained on an independent dataset of 94 CT exams, the researchers performed an observer study involving six radiologists; four were fellowship-trained cardiothoracic radiologists and two were fellows in the last few months of their fellowship. The radiologists read the studies without CAD, then immediately following with CAD, marking suspected nodules and giving a confidence rating on each.

The researchers then evaluated the performance of the radiologists using FROC and JAFROC analyses for different threshold sizes. The CAD system offered sensitivity of 78% for 3-mm nodules, 79% for 4-mm nodules, and 81% for 5-mm nodules. There was an average of 5.5 false-positive marks per nodule-free scan.

When radiologists read the cases without CAD, JAFROC analyses obtained average FOMs of 0.81, 0.83, 0.86, and 0.87 for nodule sizes greater than 3, 4, 5, and 6 mm, respectively. With the use of CAD, the corresponding average FOMs improved to 0.85, 0.86, 0.88, and 0.89, respectively.

While the increases were statistically significant for the greater than 3 mm threshold (p =0.007) and greater than 4 mm threshold (p =0.03), the gains approaches significance for nodules larger than 5 mm (p =0.055) and did not achieve significance for nodules larger than 6 mm (p =0.08).

Sensitivity was increased by 6% to 10% at different nodule diameter thresholds, Sahiner said. CAD did lead, though, to an average of 11% increase (0.55 to 0.81) in the average number of false positives per case. All of these changes were statistically significant, he said.

By Erik L. Ridley
AuntMinnie.com staff writer
December 17, 2007

CARS news: USC group develops home-grown CAD-PACS integration toolkit, June 29, 2007

Divergent research on CT lung screening sparks more debate, fewer answers, April 19, 2007

CAD performs well in lung nodule detection, February 5, 2007

Lung CT CAD boosts performance of less experienced radiologists, November 27, 2006