A PACS-integrated AI tool not only correctly identified pneumothorax on inpatient chest x-rays but also prioritized scans and improved radiologist reporting times, according to a group in Cleveland, OH.
The findings are from a “real-world” deployment and highlight the potential of AI when integrated into clinical practice, noted lead author Joshua Hunter, a medical student at Case Western Reserve University, and colleagues.
“AI algorithms in radiology capable of detecting urgent findings have gained significant traction in recent years, but the impact of these algorithms on real-world clinical practice remains unclear,” the group wrote. The research was published October 29 in Academic Radiology.
The team conducted a study that included 27,397 frontal chest x-rays consecutively obtained from August 2020 to April 2021 following deployment of the AI tool (Critical Care Suite, GE HealthCare) in the inpatient setting: 12,728 portable bedside chest x-rays were obtained using an AI-integrated scanner, which served as the AI group, and 14,669 chest x-rays were obtained using scanners without AI capability, which served as the control group.
Both types of scanners were used simultaneously in the hospital’s intensive care unit, while the scanners without AI capability were also used in non-ICU inpatient floors and in the emergency department, the authors noted.
The group conducted a receiver operator characteristic (ROC) analysis with the final radiology reports as the reference standard to evaluate the algorithm’s diagnostic accuracy for detecting the disease and Wilcoxon rank sum tests to evaluate the effect of the algorithm’s alert system for flagging suspicious cases on radiologist reporting times.
According to the findings, the area under the ROC curve (AUC) for the AI tool was 78% with a sensitivity of 60% and specificity of 97%. When selecting for moderate to large pneumothorax, the AUC increased to 93%, and sensitivity and specificity increased to 89% and 96%.
In addition, the median reporting time in chest x-rays with radiologist-confirmed pneumothorax (PTx) was reduced by 46% in those with AI integration compared to those without AI integration (100 vs. 186 minutes, p < 0.001), the group found.
“Use of an FDA-approved PTx-detecting AI tool that was integrated into our institution’s clinical PACS to generate alerts for [chest x-rays] flagged as PTx-positive demonstrated reasonable diagnostic performance and significantly improved radiology reporting times,” the researchers wrote.
Ultimately, when pneumothorax is symptomatic or large radiographically, timely decompression of the pneumothorax via needle aspiration or chest tube placement is often required, given the potential for complications like lung collapse, pleural effusion, or hemorrhage. That's why there's great value in developing methods for radiologists to efficiently identify patients with pneumothorax and to quickly communicate these findings to care providers, they noted.
“Future work in this field should include comparison of several commercially available PTx-detecting AI tools, expanding the scope to other critical findings on [chest x-rays], and prospective studies to further validate these findings,” the team concluded.
The full study is available here.
