Using AI to identify intracranial hemorrhage (ICH) on CT improves workflow in the emergency department (ED) without negatively affecting patient care, according to a study published February 9 in Radiology: Artificial Intelligence.
The findings are good news for busy emergency departments, wrote a team led by Dr. Muhannad Seyam of the University Hospital of Basel in Switzerland.
"Clinical workflow appears positively impacted by implementing an AI-based tool for detecting intracranial hemorrhage on emergently acquired CT," Seyam and colleagues noted.
In September 2020, the U.S. Centers for Medicare and Medicaid Services (CMS) announced that it would reimburse for the use of Viz.ai's AI-based technology for large-vessel occlusion detection on CT -- a decision based on "the premise and promise of enhanced clinical workflow, overall improved patient care, and lesser economic burden on the health care system," the authors wrote. This decision was renewed in August of last year.
The agency's action could pave the way for reimbursement for other AI applications, such as identifying ICH, which has high mortality and disability rates if not treated in a timely manner, according to the researchers.
Seyam's group conducted a study that assessed the effect of incorporating into their emergency department workflow a commercially available deep-learning algorithm (Aidoc) to diagnose intracranial hemorrhage on CT. The team then compared the algorithm's diagnostic performance to preimplementation diagnoses of the condition. The study included 4,450 patients who underwent emergency head CT; of those, 3,017 had exams after the AI algorithm had been implemented.
Incidence of intracranial hemorrhage was similar in both groups (pre-AI, 14.4%; post-AI, 14.3%). Regarding the AI-based algorithm's performance for detecting ICH, the team found the following:
| Performance of AI-based detection algorithm for identifying ICH | |
| Measure | Result |
| F1 score | 0.78 |
| Accuracy | 93% |
| Sensitivity | 87.2% |
| Specificity | 93.9% |
| Positive predictive value | 70.5% |
| Negative predictive value | 97.8% |
The authors noted that the algorithm had high rates of detection for ICH (97.1%) but lower rates for subarachnoid hemorrhage (80%) and subdural hemorrhage (69.2%), indicating that "a false sense of security using AI is a pitfall that must be addressed, particularly for certain subtypes of ICH and in acute care applications," they wrote.
In terms of workflow, Seyam's group found that the AI algorithm improved emergency department metrics for detecting ICH.
| Workflow effects of AI-algorithm for detecting ICH, pre- and postimplementation | ||
| Metric | Pre-AI | Post-AI |
| Report turnaround time | 60 minutes | 63 minutes |
| Communication time of an ICH finding, overall | 70 minutes | 63 minutes |
| Communication time of an ICH finding during working hours | 96 minutes | 78 minutes |
| Communication time of acute ICH | 73 minutes | 58 minutes |
| Emergency department turnaround time for ruling out ICH, overall | 175 | 164 |
| Emergency department turnaround time for ruling out ICH, during working hours | 205 minutes | 167 minutes |
The study suggests that AI could help make emergency department operations more efficient without negatively affecting patient care, but more research is needed, according to Seyam and colleagues.
"Although AI deployment indicated workflow improvement, future efforts are necessary to streamline the workflow all along the workflow chain," they concluded. "It is crucial to define a clear framework and recognize limitations as AI tools are only as reliable as the environment they are deployed in."
