A deep-learning (DL) AI algorithm designed to help clinicians diagnose COVID-19 has been validated in a study by radiologists who interpreted x-rays with and without the model, according to a study published October 16 in Scientific Reports.
A team led by Takamichi Murakami, MD, PhD, of Kobe University in Kobe, Japan, found that the DL model was more accurate than six of eight radiologist readers and that the accuracy of the readers improved when they used the AI.
“This DL model of CXRs may be useful, especially in areas where medical resources are limited,” the group wrote.
While the development of vaccines and measures to prevent the spread of COVID-19 have succeeded in reducing the number of infected people, the threat of the disease continues worldwide, the authors noted.
Many studies have shown that DL models can accurately diagnose COVID-19 on chest x-rays (including a previous study using this model), the group added. However, only a few have compared the diagnostic performance of DL models and radiologists, a step needed to determine whether the model may be clinically useful, they wrote.
To that end, the group used a previously unseen dataset consisting of 180 chest x-ray cases (60 normal, 60 pneumonia, and 60 COVID-19) from patients diagnosed with the disease using RT-PCR tests between January 24, 2020, and May 5, 2020.
Results of Grad-CAM and Grad-CAM++ for the deep-learning model: (A) normal, (B) pneumonia, and (C) COVID-19. Left column, original chest x-ray image; middle column, result of Grad-CAM; right column, results of Grad-CAM++ .Image courtesy of Scientific Reports.
Eight radiologists performed two reading sessions: one session with reference to the chest x-rays only, and the other with reference to both the chest x-rays and the results of the DL model.
The mean accuracy of the DL model on its own was 73% for identifying the correct pathology, while that of the eight radiologists without the DL model was 70%. In cases where the radiologists referred to the DL model’s findings, the mean accuracy of the radiologists was 72%, according to the findings.
“Our DL model alone showed better diagnostic performance than that of most radiologists. In addition, our model significantly improved the diagnostic performance of radiologists” the researchers wrote.
The researchers noted that because the reading sessions were conducted by radiologists with and without the DL model, the study environment simulated a practical clinical use setting. In addition, the current study of the DL model was performed using external validation, which generally produces more reliable results than internal validation, they wrote.
“Because the DL model alone had a higher diagnostic performance than the majority of the eight radiologists, it may be possible to apply the DL model to COVID-19 diagnosis on [chest x-ray] for screening,” the group concluded.
The full article is available here.
