CAD vendors hope FDA hearings will clear approval bottleneck

Mar 3, 2008

The U.S. Food and Drug Administration (FDA) today began a two-day set of hearings that could result in major changes to the way the agency regulates computer-aided detection (CAD) technology. Industry experts are hoping the hearings will clear up a logjam of delayed regulatory applications for CAD software that has been building since last year.

The FDA's Radiological Devices Panel of the Medical Devices Advisory Committee is meeting in Gaithersburg, MD, on March 4 and 5 to discuss the current state of CAD technology and the processes by which CAD should be evaluated for commercial approval in the U.S. Industry experts believe that the decisions that come from the meeting will impact the commercial development and availability of CAD products for use by radiologists.

Shrouded in secrecy

The FDA meeting has been shrouded in secrecy since its announcement in the Federal Register on February 7. "There will be a general discussion focusing on the general methodologies for CAD, including how CAD devices are used in clinical decision-making, how the devices are tested, and the information needed to properly assess their safety and effectiveness. Specific discussions relating to mammography CAD devices, colon CAD devices, and lung CAD devices" would be conducted, and "will focus on testing issues related to the different devices," the announcement stated.

The FDA didn't publish the agenda and briefing package for the meeting until February 29. No advance information was made available to any non-FDA-affiliated meeting participant, including the CAD vendors and members of industry lobbying groups preparing statements for the hearing. Representatives for virtually all medical device manufacturers involved with CAD technology are expected to attend the hearings. Click here for the meeting announcement.

The meeting comes amid a growing bottleneck of CAD regulatory approvals at the FDA. CAD product submissions during calendar year 2007, whether under the premarket approval (PMA) or 510(k) processes, have neither been approved nor denied, according to multiple industry representatives contacted by AuntMinnie.com, each of whom requested anonymity. The number of products affected is unknown.

The state of limbo within the FDA is affecting numerous issues relating to the commercialization of new CAD products, both by established companies with commercially approved products and new market entrants, according to industry sources.

There are huge financial implications as well. The Medical Imaging and Technology Alliance (MITA), a division of the National Electrical Manufacturers Association (NEMA) located in Arlington, VA, has been working for months with its members and the FDA on the issue. MITA has declined to speak publicly to any member of the press about its activities or the statement it will make at the FDA hearings until after the hearings have concluded.

The CAD vendors who talked with AuntMinnie.com were unanimous in their optimism that the hearings will produce clearly defined rules for either 510(k) or PMA submissions that are consistent based on the type of CAD application or the anatomical use of CAD. While strong differences of opinion exist about what precisely the FDA should do, all agree that the current requirements need modification.

The participants

According to the introductory statement in the official briefing packet, the FDA is requesting the advisory panel's recommendations on the data to be used to support the approval or clearance of CAD (computer-aided detection and/or diagnosis) software applied to radiological images, with emphasis on CAD devices. Because the FDA is required by statute to be least burdensome in its requirements, it is asking the panel to consider what data are essential in the context of the growth and development of the technology and its clinical use over the past 10 years.

Dr. Leonard M. Glassman, a radiologist affiliated with Washington Radiology Associates in Washington, DC, is acting chairperson of the committee. Other Radiological Device Panel participants of the hearing include four voting members (professors of radiology and radiation oncology in major academic hospitals), 15 temporary voting members (the majority of whom are professors of radiology, surgery, psychology and biostatistics), and two nonvoting members representing consumers and industry.

General questions

General questions the committee will be asked to determine include the following:

The extent to which algorithm descriptions, training dataset descriptions, standalone performance of the device on the training database, and/or stability analysis of the algorithm to training should be part of the original CAD submission, or part of subsequent algorithm updates
Appropriate constraints on the reuse of test data to balance data integrity and data collection
Recommendations for retrospective and/or prospective study designs, in addition to the Burhenne study design (Burhenne, LJW; et al, "Potential Contribution of Computer-aided Detection to the Sensitivity of Screening Mammography," Radiology, May 2000, Vol. 215:2, pp. 554-562) that has been used in the past to evaluate CAD devices
The appropriate controls for reader studies, specifically whether studies should include unaided double reading by a single reader or two readers, or reading aided with a sham CAD, in which the sham CAD randomly places the same number of marks on an image as the CAD algorithm being evaluated
The appropriateness of using standardized weighted analysis as a primary or secondary analysis of a CAD study; weighted analysis would potentially minimize the distribution of clinical variables that make it difficult to compare multiple studies of a CAD algorithm's performance
Whether different summaries of per-patient results should be considered to assess the robustness of per-patient conclusions as a unit of analysis; other unit types that are evaluated include per-region and per-lesion analysis

Issues specific to mammography CAD

Whether standalone testing using a larger database than clinical performance testing should be conducted, and if so, whether results should be stratified according to mammographic finding type, the size of mammographic findings, and/or breast composition
What are the merits of reporting standalone performance on the basis of per-lesion, per-view, per-breast, and/or per-patient analysis
What method of marking or scoring methodology should be used for standalone reporting
Whether reader performance tests should be considered in the evaluation of mammography CAD, and if so, what types of tests should be used to determine the impact of a CAD device on physician performance
If reader performance tests should be considered, what should be the required criteria, such as clinically significant size and use of receiver operator characteristic (ROC) analyses, and/or whether effectiveness analyses should be conducted separately for cancers manifesting as masses versus microcalcifications; whether reading time with and without CAD should be measured and evaluated
Whether there are specific situations in which reader performance testing may or may not be necessary, and whether the committee recommends other types of performance testing
Whether an enriched dataset for clinical evaluation testing be used due to the low prevalence of breast cancer in a screening population, and if recommended, the testing applications and content of the enriched dataset
The data required to assess the performance CAD algorithm approved for use with film-screen or a specific full-field digital mammography (FFDM) device, and whether both device standalone performance and reading performance testing in this circumstance are necessary

Issues specific to lung CAD

Defining ground truth for lung CAD devices developed for chest x-ray and chest CT, based on the possible options of various combinations of use of expert panels, pathology, CT, and PET/CT
Whether standalone performance testing should be used in the clinical evaluation of lung CAD devices, and situations in which this type of testing may be unimportant
In situations in which standalone performance testing is recommended, whether substrata such as specific nodule characteristics, co-morbidities, CT dose, or other imaging protocols should be evaluated
Whether reader performance testing should be utilized, and in what specific situations, and if so, the criteria required, including primary and per-lesion/per-region/per-patient end points, clinically significant size, use of ROC analyses, and whether reading time with and without CAD should be measured and evaluated
Other types of performance testing that should be considered
Whether an enriched dataset be used, and if so, the quantity and type of pathological and clinical characteristics that should be included
Whether the presence of other clinical conditions alter the effectiveness of the CAD function or the risk-benefit profile of a lung CAD device

Issues specific to colon CAD

Whether CT colonoscopy (CTC or virtual colonoscopy) CAD has potential clinical utility, including improved sensitivity to detect polyps of different sizes, reduced reading times, and or guiding optical colonoscopy
Defining ground truth for CTC datasets, based on the use of CTC expert review and optical colonoscopy corroboration
Whether standalone performance testing should be used, and if so, in what situations and whether substrata such as nodule characteristics, nodule pathology, co-morbidities, CT dose, or other imaging protocols should be evaluated
Whether reader performance testing is recommended, as described above
Other types of performance testing that should be considered

Future coverage

AuntMinnie.com will provide coverage of the hearing and the reaction of attendees to the discussions.

By Cynthia Keen
AuntMinnie.com contributing writer
March 4, 2008

Breast MRI CAD doesn't improve accuracy due to poor DCIS detection, February 18, 2008

Study finds that CAD improves mammography's sensitivity, February 13, 2008

Mammo CAD results show reproducibility in serial exams, January 10, 2008

UnitedHealthcare postpones CAD decision, November 27, 2007