An artificial intelligence (AI) algorithm helped radiologists from Taiwan improve their performance in detecting pancreatic cancer on CT exams, demonstrating strong performance across a range of ethnicities, according to research published online May 26 in Lancet Digital Health.
A team of researchers led by Dr. Kao-Lang Liu of Taiwan University Hospital found that their convolutional neural network (CNN) yielded sensitivity that was comparable to -- or even better than -- that of radiologists for identifying pancreatic cancer on two datasets from Taiwan. The algorithm also yielded promising results when applied to an external test set from the U.S.
"These results support that CNN can robustly recognize the CT characteristics of pancreatic cancer and thus hold great promise for developing computer-aided detection and diagnosis tools for pancreatic cancer," the authors wrote.
They compared the performance of their deep-learning model with radiologists on a local dataset that included 75 patients with pancreatic cancers and 64 controls, as well as a second local dataset of 101 patients with pancreatic cancer and 88 controls. This combined local dataset included images acquired on six CT scanners from four CT vendors.
Next, they assessed the CNN's performance on a dataset from the U.S. consisting of 281 pancreatic cancers and 82 controls.
| Performance of CNN algorithm for detecting pancreatic cancer | ||
| Radiologists | CNN | |
| Sensitivity on local dataset #1 | 94.4% | 97.3% |
| Sensitivity on local dataset #2 | 91.7% | 99% |
| Sensitivity on U.S. dataset | n/a | 79% |
| Specificity on U.S. dataset | n/a | 97.6% |
| AUC on U.S. dataset | n/a | 0.920 |
The improvement in sensitivity on the second local dataset was statistically significant (p = 0.014), as was the improvement in sensitivity (98.3% versus 92.9%) when calculated together for both local datasets.
In the combined local datasets, the radiologists missed 12 of the pancreatic cancers, 11 of which were correctly identified by the CNN. One tumor -- 1.2 cm in size -- was missed by both the CNN and the radiologist.
"The excellent performance of our CNN in all local test sets supported the notion that a well-trained CNN based on adequate training data could reliably detect pancreatic cancer and serve as a computer-aided diagnostic tool in real-world clinical practice," the authors wrote. "On the other hand, the decreased accuracy of our CNN in an external test set suggested that important differences in CT images might exist between different races and ethnicities, and such diversity needs to be included in the training data for the trained CNN to have good generalizability across diverse populations."
In other findings, the researchers noted that the CNN had 92.1% sensitivity for tumors smaller than 2 cm on the combined local dataset, but only 63.1% sensitivity on the U.S. test set.
"Our results suggest that with further improvements, a CNN-based computer-aided diagnostic or detection tool might reduce the miss rate in clinical practice and narrow disparities in resources and expertise," the authors wrote.
In an accompanying comment, Dr. Linda Chu and Dr. Elliot Fishman of Johns Hopkins University School of Medicine in Baltimore said the promising results show that using a CNN as a second reader can reduce misdiagnosis of pancreatic cancer and potentially lead to improved outcomes.
Additional external validation with datasets from multiple institutions is needed, however, according to Chu and Fishman. What's more, identifying pancreatic cancer before the subtle visual changes are apparent to a radiologist will require accumulating sufficient training data containing early subtle pancreatic cancer. That's a difficult challenge, though, because average-risk patients don't routinely receive pancreatic cancer screening, they said.
In clinical practice, radiologists also encounter various types of pancreatic cancers in clinical practice, as well as neoplastic conditions such as pancreatitis that can simulate the appearance of pancreatic tumors, according to the authors.
"Additional training will be necessary to distinguish pancreatic cancer from these other pathologies to improve the clinical utility of the model," Chu and Fishman wrote.
