An artificial intelligence (AI) algorithm can quantitatively assess the severity of COVID-19 on frontal chest x-rays, offering potential as a clinical triage and workflow optimization tool, according to research published online July 22 in Radiology: Artificial Intelligence.
A team of researchers led by first author Dr. Matthew Li and senior author Jayashree Kalpathy-Cramer, PhD, from Harvard Medical School trained a special type of deep-learning algorithm called a convolutional Siamese neural network to provide a pulmonary x-ray severity (PXS) score of patients with COVID-19.
In testing on internal and external datasets, the algorithm's PXS score correlated well with assessments by radiologists and could also help to predict if a patient would need intubation or would die within three days of admission.
"The automatic PXS score can potentially be rapidly scaled and deployed, which has important clinical applications in the COVID-19 pandemic, particularly in countries like the United States or under-resourced settings where [chest x-rays] are frequently acquired, while CT studies are relatively rarely obtained," the authors wrote.
The Siamese neural network calculates the PXS score by separately assessing two chest x-rays -- one from a patient and one known to be normal -- and then assessing the differences between the two images.
The model was pretrained using the publicly available CheXpert dataset, with subsequent training on a chest x-ray dataset of 314 COVID-19 patients. Two fellowship-trained thoracic radiologists and one in-training radiologist then each assigned the COVID-19 admission x-rays with a modified Radiographic Assessment of Lung Edema (mRALE score) ranging from 0 (no consolidation or ground-glass/hazy opacities) to 24 (complete consolidation in both lungs).
Internal testing was performed on an additional dataset of 154 COVID-19 admission chest x-rays, according to the authors. Of these patients, 92 also had a follow-up chest x-ray and these were used for longitudinal analysis. External testing was conducted on 113 consecutive admission chest x-rays from COVID-19 patients at a community hospital -- Newton-Wellesley Hospital in Newton, MA.
| Correlation of AI algorithm with average mRALE scores by radiologists | ||
| Internal dataset | External dataset | |
| Pearson correlation (r) | 0.86 | 0.86 |
| Spearman rank correlation | 0.84 | 0.78 |
In addition to finding strong correlation with radiologist mRALE scores, the researchers also concluded that the algorithm was useful for assessing longitudinal change. It yielded a Spearman's rank correlation of 0.74 when compared with the majority vote among the three radiologist raters for assessing disease changes.
Siamese neural network-based pulmonary x-ray severity (PXS) score can be used to assess longitudinal change in radiographic disease severity over time in COVID-19 patients. (A) Boxplot shows the PXS score correlates with majority vote change in pulmonary disease severity (ρ = 0.74, p < 0.001), where -1, 0, and 1 indicate decreased, unchanged, and increased severity in longitudinal chest radiograph pairs, assigned by three independent raters (two thoracic radiologists, one in-training radiologist). The boxplot boxes indicate the median and interquartile range (IQR), with whiskers extending to points within 1.5 IQRs of the IQR boundaries. (B) Examples of PXS score evaluation of longitudinal change in three patients with COVID-19. Image and caption courtesy of Radiology: Artificial Intelligence.What's more, the researchers determined that the median PXS score was higher on both the internal and external test sets in patients who were intubated or dead (PXS score = 7.9) compared with those who were not intubated (PXS score = 3.2). The difference was statistically significant (p < 0.001).
The authors noted that the PXS score, with further validation, could be incorporated into clinical treatment guidelines and used together with other clinical and lab data.
"The score could be validated for association/prediction with other outcomes, like oxygen saturation," the authors wrote. "Beyond the COVID-19 pandemic, this automated severity score could also be modified and applied to other continuous disease processes manifesting on chest radiographs, like pulmonary edema, interstitial lung disease, and other infections."
