An artificial intelligence (AI) model was able to help nuclear medicine physicians improve their performance in diagnosing bone metastases on bone scintigraphy exams in patients with breast, lung, prostate, or other cancers, according to research published online October 12 in Scientific Reports.
A team of researchers led by co-first authors Dr. Zhen Zhao and Yong Pi of Sichuan University developed a deep-learning model that yielded nearly 94% overall accuracy in classifying bone metastasis. It also helped two out of three nuclear medicine physicians in the study to achieve better sensitivity and overall accuracy.
"This model simultaneously improved diagnostic performance and time-cost for interpreting images, and the AI consulting system could potentially improve physicians' diagnostic skills especially for younger physicians who [lack] experience," they wrote.
The researchers trained, validated, and tested a convolutional neural network using 12,222 cases of technetium-99m methylene diphosphonate (MDP) bone scintigraphy exams performed on either a Discovery NM/CT 670 (GE Healthcare) or Precedence 16 SPECT/CT (Philips Healthcare) gamma camera.
On an independent test set of 400 cases (200 with bone metastasis and 200 without), the algorithm produced 93.5% accuracy, sensitivity, and specificity, as well as the following area under the curve (AUC) values:
- 0.988 for breast cancer
- 0.957 for lung cancer
- 0.955 for prostate cancer
- 0.971 for other cancers
Of the 81 (6.6%) cases that were misdiagnosed, 38 (3.1%) were false negatives and 43 (3.5%) were false positives.
The researchers also found that the AI model improved the performance of three nuclear medicine physicians, who had read the cases initially on their own and then again three days later with the assistance of the algorithm.
| Performance of AI for diagnosing bone metastases on test set | ||||||
| Nuclear medicine physician #1 | Nuclear medicine physician #2 | Nuclear medicine physician #3 | ||||
| Without AI | With AI | Without AI | With AI | Without AI | With AI | |
| Accuracy | 84.3% | 88.5% | 89% | 90.5% | 84% | 85.8% |
| Sensitivity | 74% | 81% | 85% | 86% | 78.5% | 85% |
| Specificity | 94.5% | 96% | 93% | 95% | 89.5% | 86.5% |
| Positive predictive value | 93.1% | 95.3% | 92.4% | 94.5% | 88.2% | 86.3% |
| Negative predictive value | 78.4% | 83.5% | 86.1% | 87.2% | 80.6% | 85.2% |
In comparison with the highest performances among the three physicians, the AI model had significantly higher overall accuracy and sensitivity (p < 0.001), but statistically comparable specificity. However, both physician 1 and 3 achieved better diagnostic performance from the use of AI, according to the researchers.
They also noted that the AI model took 11.3 seconds to interpret 400 cases, a 99.9% time savings compared with the 116, 140, and 153 minutes, respectively, required by the three nuclear medicine physicians.
"With further assessment and validation, this model could facilitate [diagnostic] programs and help physicians improve the diagnostic efficiency and accuracy of bone metastasis, particularly in remote or low-resource areas, leading to a beneficial clinical impact," the authors wrote.
The researchers acknowledged that in clinical practice, accurate interpretations of bone scintigraphy exams require consideration of other factors such as the patient's injury history, surgical record, characteristics on other imaging modalities, and laboratory test results. Consequently, they said they are now developing a new AI model based on analysis of fused SPECT/CT images and medical records.
