KISSIMMEE, FL - Artificial intelligence (AI) for CT imaging does not significantly improve pulmonary embolism (PE) diagnosis, suggests a June 9 presentation at the Society for Imaging Informatics in Medicine (SIIM) annual meeting.
In his presentation, Dr. Steven Rothenberg from the University of Alabama at Birmingham showed results from a study he led examining AI's use in potentially affecting miss rates and accuracy. The team found in their prospective study that while improvements were made to both, they weren't significant.
"AI improved the detection of what we consider to be clinically insignificant pulmonary emboli," Rothenberg said. "There is some room for improvement."
CT is the gold standard imaging modality when it comes to diagnosing PE. Previous research shows that both radiologists and AI algorithms are highly accurate in detecting PE on dedicated chest CT angiography. Previous research shows that AI has a sensitivity range of 0.76 and 0.95, as well as a specificity range of 0.75 to 0.96 when it comes to detecting PE.
However, most studies conducted of AI's performance in radiology are based on retrospective data.
"There are hundreds of radiology practices across the country that are using, implementing, and paying for this technology," Rothenberg said. "But there is a relative paucity of prospective validation studies."
Rothenberg and colleagues wanted to see if the miss rate of unassisted radiologists detecting PE on CT angiography would be higher compared with radiologists who were assisted by an AI algorithm for detection and triage.
They looked at data from 1,529 patients in two phases. The first phase had a radiologist and an AI triage algorithm independently evaluate consecutive CT angiography exams. Rothenberg said this data was used to establish a baseline for unassisted radiologist and AI interpretations. From there, radiologists were trained to review the results of the algorithm and triage system.
Dr. Steven Rothenberg from the University of Alabama at Birmingham showed results from a prospective study that found artificial intelligence does not significantly improve detection of pulmonary embolism from CT images. Rothenberg presented at the Society for Imaging Informatics in Medicine (SIIM) annual meeting in Kissimmee, FL.For the second phase, consecutive chest CT angiography exams were evaluated by radiologists using AI assistance for all cases. They manually reviewed reports and addendums for the presence or absence of PE.
The researchers found that the miss rate using discrepancies in both phases were significantly different. The rate by unassisted radiologists in phase I was 0.006% (3/503), higher than the 0.0001% (1/1026) seen for radiologists with AI assistance in phase I (p = 0.037). However, after further analysis with ground truth the overall miss rate trend improved from phase I to phase II, but the results were not statistically significant (p = 0.0892).
The study authors also found that in phase I and II, 12.7% (64/503) and 14.3% (147/1,026) of chest CT angiography exams were positive for PE, respectively (p = 0.395). They also found that in phase I and II, 4.97% (25/503) and 3.31% (34/1026) of exams had discrepant results between the radiologist's final report and the AI algorithm result, respectively (p = 0.126).
The AI algorithm by itself missed 24.6% of PE cases, Rothenberg said. He added that radiologists' accuracy changed from 97.8 to 98.8% and that radiologists missed 2% of clinically significant PEs.
Rothenberg, however, said he remains optimistic about AI's potential. He told AuntMinnie.com that while large medical centers that offer 24/7 care may not benefit from this, private practices with typically longer turnaround times may benefit.
"It's safe to say that radiologists' jobs are safe for now," he added with a smile.
