With lung cancer screening and follow-up come the unavoidable tasks of determining the size and location of solitary pulmonary nodules (SPN). And while a number of studies have examined the variability of reported nodule sizes, variability in the localization of nodules in 3D space has been somewhat overlooked.
Until now that is. A new study from the National Institutes of Health-sponsored Lung Imaging Database Consortium (LIDC) finds that nodule placement in the 3D space is highly variable, and that the variation depends more on the radiologist than the drawing method used to determine its spatial location.
"It's well known that there's a tremendous amount of variability in not only lung nodule identification, but in lung nodule 3D definition," said Dr. Ella Kazerooni, a professor of radiology and chief of thoracic imaging at the University of Michigan Health Service in Ann Arbor. Kazerooni discussed the study at the 2006 RSNA meeting in Chicago.
The LIDC is working with institutions across the U.S. to build a database of spiral CT lung images, and eventually establish consensus guidelines for spiral CT lung imaging.
"Because the majority of small lung nodules are not resected, a reference standard from histopathology is generally unavailable. Thus, assessing the source of variability in defining the spatial location of lung nodules by expert radiologists using different software tools as an alternative form of truth is necessary," explained principal investigator C. Meyer and colleagues in a concurrent report in Academic Radiology (October 13, 2006, Vol. 13:10, pp. 1254-1255).
The spatial location research aims to go beyond reporting the presence or absence of SPN to determine "how we can we best define the actual spatial location of the nodule in the 3D space that is the CT dataset, so when CAD developers are looking at these CT datasets they can use them to do things like volumetric 3D analysis," Kazerooni said.
The analysis, comprising 23 lung nodules in datasets imaged at five centers, did not include a detection component in its focus on localization. But once identified, the nodule location was given to the radiologists by slice number and lobe, and they were asked for the lesion volume and its spatial location.
"We wanted to examine the effects of the drawing methods as well as the radiologists on the 3D localization of nodules in space, and we used traditional volume estimates as well as probability maps to define the expert-derived spatial location," Kazerooni said.
The relative differences in performance of six radiologists, each applying three outlining methods to the task of defining the spatial extent of 23 different lung nodules, were evaluated. The variability of radiologists' spatial definitions for a nodule was measured using both volumes and probability maps (p-maps). Experimental data were analyzed using general linear models that included nested random effects.
The nodules were examined by six thoracic radiologists, all of whom were experienced in reading CT lung scans. The measurement methods included two semiautomatic methods (one from the University of California, Los Angeles, and one from Cornell University in New York City) and a manual method of outlining the lesion developed at the University of Iowa.
The researchers evaluated the relative differences in performance of the six radiologists, each of whom applied three annotation methods to the task of defining the spatial extent of 23 different lung nodules, the study authors wrote. Then the variability of radiologists' spatial definitions for a nodule was measured using both volumes and probability maps. The results were analyzed using a linear mixed-effects model that included nested random effects.
"Interpretation was done on calibrated displays ... and the readers' instructions were to draw one voxel outside of everything the reader believed to be a nodule," Kazerooni said. Then each reader and method was tested against the probability map comprised of all readers, she added.
"The closer the (individual) contour is to the white area in the middle that everybody included in the nodule volume, the higher the probability of that point being correct," she said. "And the further away from the center, the lower the probability that the nodule actually exists in that space."
For all nodules combined, volume and p-map model parameters were found to be significant at p < 0.05 for all methods, all radiologists, and all second-order interactions except one, the results showed.
The radiologists exerted far more influence on the results than the measurement method. Variables for radiologists and measurement methods accounted for 15% and 3.5% of the total p-map variance, respectively, and 40.4% and 31.1% of the total volume variance, respectively.
"Only 11% of the error was random, and the remainder of error was slightly more, contributed by radiologists than by the method itself," Kazerooni said. "So the major source of nodule segmentation variance lay with the radiologists and not the method, which was important when we identified and selected methods that would be used by LIDC to determine a nodule's spatial location."
Looking at other trends in the data and comparing volume analysis to probability maps, a mixed-effects model showed that the volume analysis results within any single reader's responses were very close, Kazerooni said. But the spread was larger in the probability maps among readers and reader combinations.
However, the probability map analysis appeared to have more power to detect differences in radiologist-method combinations.
"The reason for this is that you can have two very different contours that add up to the same volume, and it would seem to be a good map if you're comparing volume to volume," Kazerooni said. "But if you look at the probability of where the contour actually is, you're going to see tremendous differences because you're able to see positive and negative variations of the probability of a nodule being present at (a given) location."
The standard deviation of the volume measurement task appeared to be proportional to nodule volume, the authors noted.
By Eric Barnes
AuntMinnie.com staff writer
February 15, 2007
Related Reading
CAD performs well in lung nodule detection, February 5, 2007
PET, CT CAD scheme differentiates benign from malignant lung nodules, November 28, 2006
CT lung cancer screening reduces mortality, October 26, 2006
Part I: Automated CT lung nodule assessment advances, April 17, 2007
Copyright © 2007 AuntMinnie.com
