Artificial intelligence (AI) could reduce time-based costs without sacrificing diagnostic accuracy when it comes to managing thyroid nodule findings from ultrasound images, suggest findings published May 16 in JAMA Network Open.
A team led by Wen-Juan Tong from Sun Yat-sen University in Guangzhou, China, found while the aforementioned strategy could benefit senior radiologists, an "all-AI" strategy could be even more beneficial to less-experienced radiologists.
"By evaluating the association between ultrasonographic image features and AI-assisted diagnostic performance, we developed an optimized strategy to inform how radiologists with different expertise levels could benefit when integrating AI-generated recommendations," Tong and co-authors wrote.
Improving diagnostic performance
Previous studies suggest that AI can help improve diagnostic performance of ultrasound used to image thyroid nodules, with some saying that AI's performance is comparable to that of radiologists. Some researchers suggest that this could help radiologists better manage their workloads in this area.
However, Tong and colleagues noted that the interaction between AI-based clinical decision support systems and radiologists is not well understood. Along with that, they pointed out that the literature is limited on how to best reduce radiologist workloads and improve efficiency for thyroid nodule screening.
The researchers wanted to investigate imaging features that are tied to the effectiveness of AI assistance to build an optimized integration of AI decision aids for 10 radiologists with varied experience. Then, they tested the optimized AI strategy in a prospective cohort of six radiologists.
The optimized strategy considered AI's role in aiding with the workloads of both junior and senior radiologists with respect to various ultrasound features. The team wrote that this establishes positive progression in the loop of human-machine interaction. If the diagnostic sensitivity and specificity of images improved with AI-assisted performance, AI assistance is encouraged with these significant features. However, if performance does not improve with AI assistance, independent diagnosis by radiologists is recommended as long as these features are present.
Using retrospective data, the team included 1,754 ultrasound images from 1,048 patients with an average age of 42.1 years. Of the 1,754 thyroid nodules examined, 1,006 were malignant while 748 were benign.
For the prospective dataset tested on the radiologists, the team included 300 ultrasound images from 268 patients with an average age of 41.7 years. Of these nodules, 175 were malignant while 125 were benign.
Best AI strategies
The team found that the optimized AI strategy was linked to increased average task completion times for junior radiologists, but shorter average times for senior radiologists. The researchers suggested that the traditional "all-AI" strategy, which makes a diagnosis based on a cluster of imaging features, may benefit junior radiologists more.
| Task completion times for all-AI, optimized strategies | |||||
| Total cost, traditional strategy | Total cost, optimized strategy | Average cost, traditional strategy | Average cost, optimized strategy | p-value | |
| Junior radiologists | |||||
| Reader 11 | 1,108 | 1,419 | 15.2 | 19.4 | 0.02 |
| Reader 12 | 1,0121 | 1,251 | 12.7 | 15.6 | 0.001 |
| Reader 13 | 965 | 940 | 11.9 | 11.6 | 0.79 |
| Senior radiologists | |||||
| Reader 14 | 3,221 | 2,789 | 19.4 | 16.8 | 0.001 |
| Reader 15 | 2,027 | 1,621 | 12.5 | 10.0 | < 0.001 |
| Reader 16 | 1,496 | 1,410 | 9.5 | 8.9 | 0.06 |
The researchers also found no significant difference in sensitivity (range, 91%-100%) or specificity (range, 94%-98%) between the two strategies for the six readers.
However, AI assistance didn't improve examination of all imaging features. For junior radiologists, features not improved by AI included cystic or almost completely cystic nodules, anechoic nodules, spongiform nodules, and nodules smaller than 5 mm. For senior radiologists, features not improved included cystic or almost completely cystic nodules, anechoic nodules, spongiform nodules, very hypoechoic nodules, nodules taller than wide, lobulated or irregular nodules, and extrathyroidal extension.
Still, the study authors suggested that this "more cautious" approach to integrating AI into radiologist workflows could make way for safer implementation, as well as better trust by radiologists in AI tools.
"This technology will help radiologists to feel confident in the diagnostic information the model has provided and to evaluate when this information should be questioned," they wrote.
