An artificial intelligence (AI) model can help diagnose difficult cases of buckle fractures in pediatric patients, according to research presented at RSNA 2021 in Chicago.
A team of researchers led by Dr. John Zech of Columbia University in New York City designed and tested a deep-learning model for detecting wrist fractures in imaging of pediatric patients. The group found that the algorithm was highly accurate on its own but also helped residents improve their accuracy identifying fractures.
"Access to [AI] predictions significantly improved overall average resident accuracy in diagnosing from 80% to 93%," Zech said.
Studies suggest missed fractures may account for up to 80% of diagnostic errors in emergency departments. Buckle fractures of the distal radius, which are unique to pediatric patients, may specifically be misidentified due to their different response to injury and subtle presentation on imaging.
In this study, the group obtained a sample of 1,144 radiographs from 397 patients under the age of 18 (mean age was 10.1 years) performed typically in the emergency department between January 2015 and November 2019. In all, 65% of images were positive for any fracture, with 30% positive for buckle fractures. Only examinations that served as an initial evaluation for fracture were considered for inclusion.
A radiograph for the trained algorithm was considered positive if it identified at least one region with an 80% probability or higher for fracture. The maximum probability over all regions predicted by the model across images corresponding to a single examination served as the model's overall prediction.
For expert human comparison, four radiology and pediatric residents evaluated each radiograph twice, once using the original unannotated image and once using images the AI model had predicted as positive.
The model achieved an area under the curve of 0.92 and an accuracy of 88% in identifying any fracture (n = 110/125), according to results. It had a sensitivity of 88% and specificity of 89% for any fracture. In addition, the model was successful in identifying 90% of buckle fractures.
The AI model was significantly more accurate than the resident readers in the study, reaching 88% compared with 80% in identifying fractures on unannotated radiographs. However, when the readers were given access to the AI model's predictions, which included red "bounding boxes" in areas of interest, the residents improved their accuracy to 93%, the researchers found.
"When residents have access to AI predictions, they significantly improved their overall average performance," Zech said.
He noted there were several cases in which 100% of the residents accurately identified images as negative for fractures but that the AI model predicted as positive. After the readers had seen the model's erroneous prediction, however, only 50% of the readers labeled the image as negative.
"Here was a case where access to erroneous AI prediction actually degraded resident accuracy," he said.
Nonetheless, Zech said that the AI model is a promising approach for detecting pediatric wrist fractures with high accuracy, and it specifically demonstrates a strong ability to detect subtle buckle fractures.
Future studies will aim to validate the model on an external data set, as well as demonstrate the model's performance in a real-world, prospective clinical trial, he concluded.
