AI challenged to diagnose RA: Task complete

Artificial intelligence (AI) algorithms are primed to help diagnose rheumatoid arthritis (RA), says a group of U.S. experts. The team put AI to the task using x-rays in an international competition, with results published August 29 in JAMA Open Network.

A group led by rheumatologist Dr. Louis Bridges Jr., PhD, of the Hospital for Special Surgery in New York City launched an open challenge in 2019 to spur the development of AI algorithms for use in RA. The researchers fielded 173 entries from 26 teams in seven countries and found the top three finishers represent feasible, quick, and accurate methods to quantify joint damage linked to the disease.

"These algorithms alone or in combination could be incorporated into electronic health records, contributing to more informed and precise management of RA," the authors wrote.

Rheumatoid arthritis (RA) is a chronic disease characterized by inflammation in the synovial lining of joints. This inflammation causes joint space narrowing and bone erosion, which can lead to joint deformity and disability. The disease is typically diagnosed on x-rays, but identifying it can be challenging without an expert's interpretation, the authors wrote.

A number of studies have made advances in building AI models for scoring x-rays, but to date, no independently benchmarked, accessible, and automated methods to quantify RA-associated joint damage are available, the researchers added.

Thus, with grants they obtained from the U.S. National Institutes of Health and Bristol-Myers Squibb in 2019, the group launched the RAD2-DREAM Challenge as a means of catalyzing the development of potential tools. The challenge was based on so-called "DREAM challenges" developed by IBM and Columbia University researchers in 2006.

Participants were tasked with developing automated methods to quickly and accurately quantify overall RA damage (subchallenge 1), joint space narrowing (subchallenge 2), and erosion (subchallenge 3) from x-ray images of hands and feet.

To develop the algorithms, teams were provided with 674 sets of patients x-rays from two RA clinical trials. The images had been annotated by experts to indicate normal and damaged joints. Sharp-van der Heijde scores (SvH), or "sharp scores," were used as measures of erosion and joint space narrowing on the images on a scale from 0 to 10.

Representative radiographs showing joints without disease and joints with mild and severe damage due to rheumatoid arthritis. Red arrows indicate areas of erosion; yellow arrows, areas with joint space narrowing. Image courtesy of JAMA Open Network.

Based on root mean square error (RMSE) analysis, which measured how closely the estimated values from the algorithm matched the expert SvH scores, the winning algorithms produced sharp scores very close to those of the experts, according to the findings.

The top-performing algorithm (submitted by "Team Shirin") for measuring overall RA damage achieved an RMSE of 0.44; the winning algorithm for measuring joint space narrowing (submitted by team "HYL-YFG") achieved a RSME of 0.38; and the winning algorithm for measuring erosion (submitted by team "Gold Therapy") achieved a RSME of 0.43.

In addition, the researchers tested the algorithms using a dataset independent of the training set used in the competition, which confirmed their performance, the authors wrote.

"The RA2-DREAM Challenge resulted in the development of algorithms that provide feasible, quick, and accurate methods to quantify joint damage in RA," the researchers stated.

There was skin in the game. Team Shirin, led by Lars Ericson of the Catskills Research Company in Charlotte, NC received $6,000; team HYL-YFG, led by postdoctoral researcher Hongyang Li and associate professor Yuanfang Guan, PhD, of the University of Michigan in Ann Arbor, received $14,000; and team Gold Therapy, led by Dr. Ariel Yehuda, PhD, of Clalit Health Services in Jerusalem, Israel, also nabbed $10,000.

Ultimately, the SvH method is currently the generally most accepted scoring system for RA, yet the approach is time consuming, labor intensive, and requires specialized training, the researchers noted.

Conversely, after clinical validation, such algorithms as were developed for RA2-DREAM Challenge could be integrated into hospital electronic health record (EHR) systems for reliable, efficient monitoring of joint damage progression, the authors suggested. Another use could be for researchers studying clinical outcomes of RA who would benefit from an automated method to assess the degree of joint damage quickly, accurately, and reproducibly, they wrote.

Importantly, in such cases, the use of AI algorithms could lead to better therapeutic decisions for patients, such as changing disease modifying antirheumatic drugs that are not controlling damage detected via x-rays, they wrote.

"Such algorithms have great potential for improving outcomes in patients with RA and other chronic forms of arthritis," Bridges and colleagues concluded.