Pursuing virtual over optical colonoscopy as the primary colorectal cancer screening tool would likely raise screening costs without saving additional lives, at least in the short term, researchers reported this month in the Canadian Medical Association Journal.
The gastroenterologist-led study from the University of Calgary in Alberta, Canada, and the Medical University of South Carolina in Charleston, predicted that the handful of lives saved in the prevention of colonoscopy-associated perforations would be more or less offset by additional cancer deaths resulting from missed lesions in VC. As for costs, a presumed need for more frequent VC compared to colonoscopy was one factor that contributed to a higher cost profile for the virtual exam in addition to other factors.
The researchers acknowledged that the chosen three-year time window was too short to provide an accurate long-term picture of the costs and benefits of either modality, but emphasized that a lack of study data would render any longer-term projections too speculative to be accurate.
Commenting on the study, health policy analyst David Vanness, Ph.D., told AuntMinnie.com that on the whole, the study appeared to be fairly well designed and executed. Yet he disputed the accuracy of the conclusions, as well as some of the data used to calculate the results. Based on the study's short time frame, a lack of available data, and shortcomings in methodology, clinical results could differ significantly in the long term, he said.
"Their general conclusion should be read as 'it's too early to tell' rather than 'virtual colonoscopy is not cost-effective,'" stated Vanness, who is from the department of population health sciences at the University of Wisconsin Medical School in Madison.
The study
The authors noted in their CMAJ paper that earlier studies on VC's cost-effectiveness were already outdated.
"Previous cost-effectiveness analyses in the United States have concluded that a screening strategy employing CT colonography (CTC or VC) is more costly than one using colonography," wrote lead investigator Dr. Steven Heitman, a gastroenterology fellow at the University of Calgary, along with colleagues Dr. Braden Manns et al, and Dr. Joseph Romagnulo from the Medical University of South Carolina.
"These studies are limited by outdated CT test performance characteristics and assumptions, such as a CT rescreening interval comparable to that of colonoscopy, which is possibly inappropriate," they wrote. "Furthermore, the results of an economic valuation from one country are not necessarily transferable to another. To inform Canadian health policy, we performed an economic evaluation comparing the cost-effectiveness of CT colonography with that of colonoscopy for colorectal cancer screening" (CMAJ, October 11, 2005, Vol. 173:8, pp. 877-881).
The group used decision analysis software (Tree Age Software, Williamstown, MA) to construct a model comparing virtual to optical colonoscopy for screening average-risk patients age 50 years and older.
A three-year period was chosen for the analysis based on three considerations: the risk of cancer after therapeutic colonoscopy (due to missed lesions), the natural history of polyps 10 mm or larger, and the fact that the follow-up VC interval is unlikely to be shorter than three years, the authors wrote.
The model assumed that polyps 5 mm and larger would be referred for colonoscopy and polypectomy if confirmed. Colonoscopy-related deaths related to perforation and cancer deaths from missed adenomas were also calculated. Only VC studies using segmental unblinding were used for comparison, which limited the comparison to two large trials. For colonoscopy, back-to-back colonoscopy results were also used to assess test accuracy.
The baseline prevalence of colorectal cancer was assumed to be 0, while the prevalence of polyps 6-9 mm and 10 mm and larger were averaged from available VC studies. Natural history data were used to estimate the three-year risk of cancer after a missed adenoma (polyps < 10 mm 0.9%; ≥ 10 mm 1.5%). National Polyp Study results were used to predict that five-year survival for patients with cancers discovered after three years would be 90%. Based on 30 years of data, the risk of colonoscopic perforation and complications were calculated at 0.99% and 0.24%, respectively, with a mortality rate of 4.9%.
Indirect costs were $71.04 for VC and $213.72 for colonoscopy, owing to the need for sedation and more time off work in the latter procedure. A potential increase in screening compliance with VC was calculated along a wide range of 0% to 50%.
"Increasing the test performance of CT colonography to that reported by (best-case scenario) Pickhardt and colleagues (study) (New England Journal of Medicine, December 4, 2003, Vol. 349:23, pp. 2191-2200) resulted in an incremental cost per life gained of $4.11 million, or $220,000 per life gained," Heitman and colleagues wrote. "Decreasing its test performance to that reported by (worst-case scenario) Cotton and associates (JAMA, April 14, 2004, Vol. 291:14, pp. 1713-1719) yielded a cost savings of $4.51 million for the CT colonography strategy, which reduced the number of colonoscopies performed. However, this was accompanied by 2.27 extra cancer deaths compared with the colonoscopy strategy."
In the U.S., $50,000 per life year gained is typically considered a reasonable benchmark.
Adjusting the input data had a big effect on the results of course. Decreasing the cancer risk of a missed adenoma of 6-9 mm gave VC a cost per life year gained of $42,900. Increasing the risk of colonoscopy perforation to 0.2% lowered the cost of a life year gained with VC to $18,200, and increasing the risk of death from perforation to 14% lowered the life-year cost of VC to $2,130.
Including the indirect costs (e.g., missed work), VC saved $6.15 million, but at a cost of 0.33 extra deaths. If VC were to raise screening compliance to 50%, overall deaths would be fewer but at an incremental cost per life year.
"It is difficult to know what the test performance characteristics of CT colonography will ultimately be in routine clinical practice," the team wrote. "However, even if they rival those of colonoscopy, we have shown that the cost of a CT colonography strategy prohibits recommending it as a primary method for colorectal cancer screening. Even in the best-case scenario, the cost per life year gained was unattractive by conventional standards."
The results remained robust when the malignant potential of adenomas missed with VC were considered, and if anything, may be higher than estimated, the group wrote, citing the National Polyp Study, which pegged the three-year risk of malignancy for missed polyps at 14%, rather than the 1.5% estimated by Heitman's team. Moreover, the estimated 0.99% risk of perforation from colonoscopy was just 0.5% in one large colonoscopy trial. Finally, they wrote, data are inconclusive as to whether VC will actually increase compliance rates, further driving up the costs of each life year saved.
The limitations related to the short three-year time period, the assumption that all missed carcinomas would be early-stage. And if patients with lesions smaller than 5 mm on VC were in fact sent to colonoscopy, the costs and related deaths would rise even further.
"CT colonography does not appear to be cost-effective for primary colorectal cancer screening in Canada," they concluded. "CT colonography has a potential role in centers where the risks of colonoscopy are high, or in patient populations with high operative mortality."
The dispute
The University of Wisconsin's Vanness and study author Dr. Joseph Romagnuolo had a chance to advance their divergent opinions in e-mails to AuntMinnie.com; their remarks are excerpted at length below. It should be noted that Vanness had two opportunities to respond by press time, while Romagnuolo had only one.
Vanness praised the study authors' acknowledgement that their results are "highly sensitive to a number of unknowns -- in particular, test performance, perforation risk, procedure cost, and assumptions about progression of undiscovered lesions," adding that mining multiple studies using different techniques makes it difficult to produce reliable conclusions.
"In large part, the wide variation in reported test performance is one of the main reasons why we're looking forward to the results of the National CT Colonography Trial (ACRIN #6664)," Vanness wrote. "In particular, ACRIN #6664 seeks to address the issue about whether primary 3D reads outperform primary 2D reads -- an issue that seems to be at the heart of the effectiveness (and cost-effectiveness) issue."
Romagnuolo, an associate professor of medicine and director of gastroenterology and hepatology research at the Medical University of South Carolina, responded that the trial was designed to minimize the impact of unknowns on the outcome.
"Although Dr. Vanness points out correctly that our sensitivity analysis reveals that there are multiple important unknowns to consider, it is nonetheless robust to these unknowns," Romagnuolo wrote. "The best-case scenario for (VC) accuracy still gives an unreasonable ICER (incremental cost-effectiveness ratio) for (VC) of $200,000; only when you assume unrealistically high perforation rates (1 in 500), or unrealistically high perforation-related mortality rates (almost 15%), does that ICER come into a reasonable range (< $40,000/life year gained). Therefore, the model's conclusions are robust to plausible ranges of these variables (including VC's accuracy)."
With regard to the study model, Vanness wrote that it is unclear what test performance characteristics were used for standard colonoscopy, but noted that the colonoscopy adenoma miss rates determined from segmental unblinding in the Pickhardt et al were substantial.
"Perhaps the model's most significant limitation is that it is not a fully 'dynamic' model (like a Markov model or other discrete event model)," Vanness stated. "As the authors admit, the model fails to consider costs and consequences beyond those occurring at three years. Some of these excluded costs and consequences may bias their analysis."
Romagnuolo responded that Markov models simply cannot be applied accurately in the presence of so many unknowns.
"Markov models are useful to model changes in health states over time, but only if the transition probabilities between health states can be estimated with some degree of accuracy," Romagnuolo told AuntMinnie.com. "The problem with colon cancer is that a number of unknown transition probabilities exist -- what is the probability of a stage 1 cancer becoming stage 2 each year? What is the annual probability of advancing to nodal disease? And then from nodes to metastatic disease? What is the natural history of a polyp beyond three to five years? Although Markov models have been published on this subject, these assumptions are never documented in their summary tables for a reason -- they are guessed at. In a Markov model, these inaccuracies snowball with time. Secondly, lifelong models are unnecessary for a technology that will either outdate itself with better technology, or will be outdated by competing technology, in three- to five-year windows. Society has to be able to make short-term economic decisions...."
Whether data are available or not, long-term results do matter, Vanness responded.
"Rather than defend multiple, published dynamic models against these charges, I simply note that criticizing the dynamic modeling approach does not exonerate the authors' excessively short time frame," Vanness wrote. "The authors state that 'society has to make short-term economic decisions.' This is no doubt true. However, that does not mean that long-term economic consequences are irrelevant. In short, the choice of a three-year window does not place both strategies on equal footing, because it ascribes all of the harm to CT of missed 6-9 mm lesions becoming invasive cancer (which is based on very limited data) while failing to count the considerable surveillance costs engendered when colonoscopy finds them."
"There is no uniform screening interval screening for CT," Romagnuolo wrote. "(VC) is far more expensive than barium enema and reportedly more accurate, so it really is unlikely that the five-year barium enema interval will not be extended for CT. If, as Pickhardt has suggested, the accuracy of CT is close to that of colonoscopy for large polyps, there would be no theoretical reason why the screening interval should not be the same. If it is made shorter, CT screening strategies' costs will markedly rise. Because of this uncertainty, we placed the idea of screening intervals aside, stuck to the hard data for natural history of polyps (which does not extend out as far as 10 years), and placed both strategies on equal footing with an analysis at three years."
"The authors' rationale for choosing three years as the window for their study seems to be the desire to 'place both strategies on equal footing' and to 'stick to the hard data for natural history of polyps,'" Vanness stated. "I find it interesting that the authors choose to trash Markov models because of their wild inaccuracies, but then rely on natural history data that is hardly illuminating. It should be noted that their data for progression of 6-9 mm polyps to cancer within three years (0.9%) came from a single study of 116 patients (Hofstad et al, 1996), one of whom developed colorectal carcinoma, which in the original abstract was noted 'may be evolved from a previously overlooked polyp.'"
"The uncertainty behind the model's estimate of 0.9% is enormous (and could indeed be essentially zero)" Vanness continued. "Furthermore, that same study by Hofstad et al noted that polyps 5-9 mm showed a 'tendency to net regression in size, both for adenomas and hyperplastic polyps' and concluded that 'follow-up of unresected colorectal polyps up to 9 mm is safe.' The importance of this is very significant, because cost-effectiveness of CT colonography in the Heitman et al (study) was extremely sensitive to the risk from missed 6-9 mm adenomas. When the authors set that rate to zero, CT colonography 'had an incremental cost per life year gained of $42,900.' This is within the established range of acceptable cost-effectiveness."
"Even the data for 10 mm and greater lesions came entirely from the Stryker et al (Mayo) study published in 1987 of 226 individuals with follow-ups ranging from as little as two to as much as 20 years," Vanness wrote. "In order to obtain estimates for the 2.5% conversion rate at five years, Stryker et al used an actuarial model. In the paper under review, Heitman and colleagues made a further interpolation to obtain a base-case three-year rate of 1.5% (Table 1). Given the huge range of follow-up, the small number of observed individuals and the need for an actuarial model to interpolate results (which were then interpolated again!), the uncertainty behind the estimate is substantial. To their credit, Heitman et al do use very wide ranges for their confidence intervals -- ranges which admit the possibility that CT colonography is cost-effective. Probabilistic sensitivity analysis would perhaps have been a good exercise."
"Although surveillance costs for colonoscopy may be higher, it is because it is more accurate, and will pick up adenomas with higher frequency," Romagnuolo stated. Missing polyps is clearly cheaper -- the more insensitive you assume CT is, the cheaper the CT strategy becomes; but the number of deaths from missed cancer rises. The real-life CT costs will likely also be higher, as it is unlikely that doctors and patients will always ignore < 5-mm polyps (for which CT's specificity is poor, with many false positives), and incidental extracolonic findings on CT cost money to investigate."
"Dynamic models in colon cancer screening also make CT look better than it really is, because all people with polyps discovered are thereafter surveyed with colonoscopy, thereby effectively borrowing the effectiveness of colonoscopy in up to a third of the virtual patients choosing CT in the model," Romagnuolo wrote.
In the study, the recommended rescreening interval for colonoscopy appears to be 10 years, so some of the lesions missed by colonoscopy that did not progress to cancer within the three years of the model would progress in years four to nine, Romagnuolo stated.
"The model does not count those misses against colonoscopy," Vanness wrote. "Since the screening interval for CT colonography would presumably be shorter, there would be more opportunities to detect a missed lesion within the same 10-year period. Indeed, if the lesion is growing, the sensitivity of CT colonography to that lesion may even increase over time. In short, under current screening guidelines, a false-negative colonoscopy essentially gives 10 years of false reassurance; a false-negative CT colonography would presumably give less."
Finally, the study model does not seem to include surveillance costs triggered by positive findings, which are likely to be higher for colonoscopy, Vanness wrote. And he suggested that a true comparison of screening options should also include other approved indications such as flexible sigmoidoscopy and the fecal occult blood test (FOBT).
"In short, the conclusion of the study is likely correct: it's probably too early to tell whether CT colonography should be reimbursed as a primary screening strategy," Vanness wrote. "But there are too many unknowns at this time (and factors unaccounted for in this model) to conclude that CT colonography could not be cost-effective (or to have much confidence in the specific estimates from this study). Hopefully, results from ACRIN #6664 will help shed light on this crucial question."
"I agree," Romagnuolo stated," that a national study of CT colonography technique would be important to document the national benchmarks in accuracy and technology of CT colonography, but even if the results are as good as the best currently in the literature now, the ICER for CT remains too high when compared to colonoscopy."
By Eric Barnes
AuntMinnie.com staff writer
October 31, 2005
Related Reading
Good study/bad study: VC experts seek keys to success, April 20, 2005
What virtual colonoscopy misses might not matter, August 18, 2004
CT colonography sensitivity low, April 14, 2004
Duke VC trial results disappointing, November 1, 2004
Group credits 3-D reading for best-ever VC results, October 15, 2003
Copyright © 2005 AuntMinnie.com
