A group of Italian researchers found the performance of radiologists for detecting COVID-19 pneumonia on chest x-rays grew over time as they gained experience -- a finding that even applied to those with less experience reading x-rays. They shared their results online September 10 in the European Journal of Radiology.
After retrospectively assessing the diagnostic performance of chest x-rays performed at their university hospital during the height of the pandemic, a team of researchers led by first author Dr. Andrea Cozzi and senior author Dr. Francesco Sardanelli of the University of Milan found that the radiologists with less than 10 years of experience reading chest x-rays yielded the same 89% sensitivity level as a group with more-experienced radiologists.
Although the most-experienced radiologists produced higher specificity, both groups improved their sensitivity and accuracy over the six weeks of the study.
"The increase in accuracy over time strengthens [chest x-ray's] role as a first-line examination in suspected COVID-19 patients," the authors wrote.
To assess the real-world diagnostic performance of chest x-ray interpretations during the COVID-19 pandemic, the researchers reviewed all patients who had presented to the emergency department at their university hospital between February 24 and April 28 and who had received reverse transcriptase-polymerase chain reaction (RT-PCR) testing and an anteroposterior bedside chest x-ray within 12 hours of admission.
The team then compared the reported chest x-ray results with a reference standard consisting of RT-PCR results as well as information obtained via phone calls to patients who had negative RT-PCR results but did not have a repeated swab test during their hospitalization.
The researchers then assessed diagnostic performance, both overall and over the first and last three weeks of the study. To assess the impact of radiologist experience, they also grouped the seven radiologists in their department by experience levels; three radiologists had less than 10 years of chest x-ray reading experience, while a second group of four had more than 10 years of experience.
Both groups of radiologists produced similar overall sensitivity, but the less-experienced group had lower specificity. In addition, both groups of radiologists improved their sensitivity and accuracy over the course of the pandemic, albeit at the cost of lower specificity.
| Impact of radiologist experience on performance detecting COVID-19 on chest x-rays | ||||||
| < 10 years of chest x‑ray experience (overall) | > 10 years of chest x‑ray experience (overall) | < 10 years of chest x‑ray experience (first half of outbreak) | > 10 years of chest x‑ray experience (first half of outbreak) | < 10 years of chest x‑ray experience (second half of outbreak) | > 10 years of chest x‑ray experience (second half of outbreak) | |
| Sensitivity | 89% | 89% | 81.5% | 80.6% | 93.5% | 92.7% |
| Specificity | 40.7% | 66% | 46.7% | 74% | 33% | 58% |
| Accuracy | 76% | 83.7% | 69% | 78.4% | 81% | 86% |
The researchers noted that as this was a real-world data study, the results rely on a practical dichotomization of chest x-ray reports. As a result, caution should be exercised in considering the potential generalizability of their results, especially in cases of a nontypical chest x-ray for SARS-CoV-2 pneumonia, they said.
