An artificial intelligence (AI) algorithm can predict -- from electronic medical record (EMR) data -- if a patient is likely to have a pulmonary embolism (PE) prior to receiving CT pulmonary angiography, potentially avoiding unnecessary scans, according to research published online August 7 in JAMA Network Open.
A research team led by Imon Banerjee, PhD, of Stanford University developed its pulmonary embolism result forecast model (PERFORM) to provide pre-PE CT risk scores based on analysis of EMR data such as demographics, vital signs, diagnoses, medications, and laboratory test results. In testing, the team's deep-learning model significantly outperformed three popular clinical scoring models for predicting a positive PE finding.
"This approach might be used as an automated clinical decision-support tool for patients referred for CT PE imaging to improve CT use," the authors wrote.
Although clinical decision-support (CDS) rules have been developed based on PE risk-scoring models to estimate pretest probability, they are underused and tend to underperform in practice -- resulting in persistent overuse of CT for PE, according to the researchers. As a result, the group sought to develop a machine-learning model to provide a patient-specific risk score for predicting imaging outcomes.
To train and validate potential models, the researchers first gathered 3,397 annotated CT exams performed for PE from 3,214 unique patients from Stanford hospitals and clinics. They then assessed the performance of several different types of models, including ElasticNet, a traditional machine-learning algorithm; an artificial neural network model; and other machine-learning approaches.
External validation of the models was also performed using 244 annotated CT exams from 240 unique patients from Duke University Medical Center. Although the ElasticNet model performed better in testing on a holdout sample of 340 CT exams, the deep-learning model proved to be more generalizable. PERFORM was also significantly more accurate than three clinical scoring models -- Wells, pulmonary embolism rule-out criteria (PERC), and revised Geneva (rGeneva) -- when compared on randomly selected samples of 100 outpatients from Stanford and 101 outpatients from Duke.
| Area under the curve for predicting a positive PE CT study | |||||
| Wells | PERC | rGeneva | ElasticNet machine-learning model | Artificial neural network model | |
| Outpatients from Stanford | 0.48 | 0.51 | 0.53 | 0.73 | 0.81 |
| Outpatients from Duke | 0.51 | 0.60 | 0.47 | 0.74 | 0.81 |
"The neural network model PERFORM possibly can consider multitudes of patient-specific risk factors and dependencies in retrospective structured EMR data to arrive at an imaging-specific PE likelihood recommendation and may accurately be generalized to new population distributions," the authors wrote.
Using conservative cut-off scores, PERFORM would have avoided 67 of 340 studies at Stanford and 147 of 244 studies at Duke, resulting in a 78% improvement in positive CT yield at Stanford and 40.2% at Duke. What's more, false negatives would have occurred less frequently than the current best practice, according to the researchers. They also noted that patients with PE who had a high positive PE pretest prediction score could benefit from improved clinical outcomes by receiving early treatment.
