The performance of the winning algorithms of RSNA’s 2022 Cervical Spine Fracture AI Challenge highlights the potential of such competitions to move the field forward, according to an article published January 4 in Radiology: Artificial Intelligence.
“The performance of the top eight algorithms in the RSNA Cervical Spine Fracture Detection competition appears to exceed all previously reported study-level algorithm performances of individually trained models in the literature,” wrote corresponding author Luciano Prevedello, MD, of Ohio State University in Columbus.
The RSNA 2022 Cervical Spine Fracture AI Challenge invited participants to develop AI models that could accurately detect, identify, and localize fractures in the cervical spine on CT scans and was open to the public on Kaggle from July 28 to October 27, 2022.
A total of 1,108 competitors comprising 883 teams worldwide participated in the challenge, with 12,871 entries submitted. The eight top-performing algorithms were selected based on their weighted log-loss performance in the private test set, RSNA said.
The mean area under the receiver operating curve (AUC) value of the top eight algorithms of the RSNA Cervical Spine Fracture AI Challenge was 0.96, compared with the highest previously reported value of 0.85.
The mean F1 score of the top eight algorithms of the competition was 90%, compared with the highest reported value of 81% from previous literature for a machine learning algorithm, according to the results.
“While the outcome of the RSNA competition is promising, it is important to note that the research still remains in a very early stage and more rigorous studies are needed to assess the potential clinical utility of such algorithms in a clinical environment,” the authors noted.
A link to the full article can be found here.
