A deep-learning algorithm performs comparably to radiologists and may improve radiology workflow for determining whether a thyroid nodule should be biopsied, according to a new study published online July 9 in Radiology.
The results address a key concern about managing thyroid nodule findings identified on ultrasound: interreader variability, wrote a team led by Mateusz Buda of Duke University School of Medicine in Durham, NC.
"Imaging with ultrasound remains an accurate method to guide recommendation for management of thyroid nodules, although interpretation variability and overdiagnosis represent continual challenges," the group wrote. "Some of the challenges of thyroid ultrasound interpretation and reporting data systems such as [the American College of Radiology (ACR) Thyroid Imaging Reporting and Data System (TI-RADS)] represent problems that may be solved through deep-learning applications."
Buda and colleagues developed a deep-learning algorithm that performed three tasks: detecting nodules, predicting malignancy, and stratifying risk level of the nodule. The study included 1,377 thyroid nodules imaged by ultrasound between August 2006 and May 2010. All nodules had complete imaging data and cytologic or histologic diagnoses. Of the 1,377 nodules, 10.3% were malignant.
The researchers trained the algorithm to provide biopsy recommendations using ultrasound images of 1,278 nodules, then used 99 ultrasound images as a test set to compare its sensitivity and specificity to the consensus interpretations of three ACR TI-RADS committee experts and nine other radiologists. Of the training nodules, 9.9% were malignant; of the test set nodules, 15% were malignant.
Buda's group found that the deep-learning algorithm's sensitivity for biopsy recommendation was the same as the consensus results of the experts and higher than five of the nine radiologists. Its specificity for biopsy recommendation was similar to expert consensus and higher than seven of the nine radiologists.
| Comparison of deep-learning algorithm to radiologist readers for evaluating thyroid nodules | |||
| Performance measure | 9 radiologist readers (mean values) | 3 ACR TI-RADS experts | Deep-learning algorithm |
| AUC* for distinguishing between benign and malignant nodules | 0.82 | 0.91 | 0.87 |
| Sensitivity | |||
| For biopsy recommendation | 83% | 87% | 87% |
| For follow-up | 97% | 100% | 93% |
| Specificity | |||
| For biopsy recommendation | 48% | 51% | 52% |
| For follow-up | 34% | 40% | 38% |
The study results add to a growing body of evidence that shows the usefulness of deep learning when applied to thyroid ultrasound, according to Buda and colleagues.
"The most valuable aspect of the deep-learning algorithm is the ability to improve specificity of thyroid nodule biopsy recommendations," the group wrote.
The algorithm could make a difference in clinical practice, the researchers noted.
"First, for a given image, our algorithm will always provide the same prediction. Therefore, it will eliminate a substantial interreader variability that has been observed for this task even when the ACR TI-RADS system is used. Second, the algorithm could reduce the time required for interpretation of thyroid nodules, which puts some strain on radiology departments. Finally, deep learning may perform better than some radiologists who interpret thyroid ultrasound images in clinical practice," the group concluded.
