The sensitivity of computer-aided detection (CAD) algorithms will change depending upon the criteria used to evaluate the precise location of a mark on a suspicious finding. Thus, marking criteria should be specifically defined, both by CAD product vendors and authors reporting study results, according to Dr. Feng Li, a radiologist at the University of Chicago Kurt Rossmann Laboratories for Radiologic Image Research.
The University of Chicago recently conducted two studies to evaluate the number of actual true-positive detections of lung cancers by CAD markings on chest radiographs versus accidental markings. Using version 1.0 of OnGuard CAD software (Riverain Medical, Miamisburg, OH), one study team processed 34 posteroanterior (PA) chest radiographs of patients whose nodular cancers were missed at original interpretation between 2001 and 2004.
The nodule sizes ranged from 6.8 mm to 22.7 mm (mean, 14.5 mm), according to Li. Two different criteria were used to indicate positive detection: detection-in-nodule, in which the center of the circle coincides with the location of the nodule or part of the nodule; and nodule-in-circle, in which the nodule is located partially or completely within the circle, Li said. He presented the research at the 2007 RSNA meeting in Chicago.
Based on the strictest detection-in-nodule criteria, sensitivity for nodule detection was 35%, with an average of 5.9 false-positive marks, Li said. Sensitivity increased to 47%, with 5.7 false positives, when the nodule had to be completely within the circle. When only a portion of the nodule needed to be within the mark, sensitivity increased by another 12%, with 5.6 false positives.
In the second study, researchers evaluated a new database of 47 PA chest radiographs showing solitary lung nodules ranging from 8 mm to 30 mm (mean, 19.9 mm), using versions 1.0 and 3.0 (submitted in August 2007 for U.S. Food and Drug Administration clearance).
For this study, the detection-in-nodule criteria included nodules that overlapped the center of a circle mark.
"We believe that this reflects a true-positive mark in clinical use," said Dr. Heber MacMahon, another co-author of the studies. "Where we feel that sensitivity may be overstated is when CAD misidentifies a normal blood vessel and marks this, and a cancer happens to be located on the edge of that circle."
By broadening the criteria of detection-in-nodule, sensitivity rose to 55% with version 1.0 of the software and rose to 62% with version 3.0. Nodule-in-circle sensitivity similarly increased to 70% with version 1.0, but declined by 2% with version 3.0. With version 3.0, the average number of false-positive marks dropped by 50% to an average of 2.8 false positives per image.
"When evaluating ... CAD systems, not just for lung CAD, it is important to understand how their apparent accuracy can be influenced by the specific criteria that are used to determine sensitivity and specificity," Li said.
"For this CAD application, it makes sense that a radiograph with fewer markings will be less susceptible to accidental detections," MacMahon said. "If you have almost six circles on every chest x-ray, the chance of hitting a cancer becomes quite high."
By Cynthia E. Keen
AuntMinnie.com contributing writer
March 20, 2008
Related Reading
CAD provides mixed benefits for DR lung exams, March 8, 2008
DR image processing produces mixed CAD results, February 12, 2008
Chest x-ray lung CAD reimbursement picks up, October 24, 2006
X-ray CAD aids in early lung cancer detection, June 22, 2006
NCI CAD database program gathers momentum, March 16, 2006
Copyright © 2008 AuntMinnie.com
