There’s a notable lack of consensus on key aspects of AI bias in radiology, and this problem has prompted a radiologist-led team to outline best practices for measuring and mitigating the risks.
Lead author Paul Yi, MD, from the department of radiology at St. Jude's Research Hospital in Memphis, TN, and a group of radiologists, computer scientists, and informaticists, including senior author Jeremias Sulam, PhD, from Johns Hopkins University in Baltimore, MD, have been troubleshooting the issue of AI bias in radiology. A commentary the group crafted was published May 20 in Radiology.
The authors drew on lessons learned from a previous study Yi led that evaluated 23 publicly available chest radiograph datasets and found that only 17% of study participants reported race or ethnicity.
The status quo for demographic reporting has been suboptimal, according to Yi.
“Despite the significant attention this topic receives, there’s a notable lack of consensus on key aspects such as statistical definitions of bias, how demographics are categorized, and the clinical criteria used to determine what constitutes a ‘significant’ bias," he said in a statement released by the RSNA.
The term "bias" is used in the context of demographic fairness, which reflects differences in metrics between different demographic groups, the authors noted, in their paper addressing three key areas where pitfalls occur:
- Medical imaging datasets
- Demographic definitions
- Statistical evaluations of bias
The work provides a roadmap for more consistent practices in measuring and addressing bias, with the goal of ensuring that AI supports inclusive and equitable care for all people, according to Yi, whose team noted the following:
- Medical imaging datasets should report demographic variables such as age, sex, race, and ethnicity, as a standard practice, within patient privacy regulations. The authors also recommended collecting common confounding features in medical imaging datasets, such as imaging view, hospital site, inpatient versus outpatient imaging, and scanner brand and model.
- Definitions of demographic groups are a challenge because demographic categories such as gender or race are not biological variables but self-identified characteristics, this research noted. "Although they may seem like static concepts, demographic categories are fluid," the authors wrote.
- Statistical evaluation of bias raises the question of how conclusions will be used to inform health policy at the hospital level, nationally, or internationally.
![Example of learning to make a diagnosis based on spurious correlation [e.g., laterality marker], rather than the disease or condition itself. (A) Images from a deep learning (DL) model in radiology that can learn to identify confounding features related to bias and unfair predictions, including laterality markers (image annotations indicate the side of the body being viewed [right vs left]) to identify the hospital at which a chest radiograph was obtained. (B) Image from a DL model that can make a diagnosis of radiographic abnormality on extremity radiographs, also known as shortcut learning. A. Images adapted and reprinted from reference 57, an open-source article, published under the Creative Commons license (CC BY 4.0). B. Reprinted, with permission, from reference 59. All courtesy of the RSNA.](https://img.auntminnie.com/mindful/smg/workspaces/default/uploads/2025/05/101148-radiol241674figure3.kYHaIAI5tS.png?auto=format%2Ccompress&dpr=2&fit=max&q=70&w=700)
Example of learning to make a diagnosis based on spurious correlation [e.g., laterality marker], rather than the disease or condition itself. (A) Images from a deep learning (DL) model in radiology that can learn to identify confounding features related to bias and unfair predictions, including laterality markers (image annotations indicate the side of the body being viewed [right vs left]) to identify the hospital at which a chest radiograph was obtained. (B) Image from a DL model that can make a diagnosis of radiographic abnormality on extremity radiographs, also known as shortcut learning. A. Images adapted and reprinted from reference 57, an open-source article, published under the Creative Commons license (CC BY 4.0). B. Reprinted, with permission, from reference 59. All courtesy of the RSNA.
In an accompanying editorial, Melissa Davis, MD, of Yale School of Medicine in New Haven, CT, said the article underscores the importance of a collective effort to address the challenges posed by bias and fairness in AI.
"By fostering collaboration between clinicians, researchers, regulators, and industry stakeholders, the health care community can develop robust frameworks that prioritize patient safety and equitable outcomes," Davis wrote. "Although many questions remain unanswered, the authors provide a valuable foundation for future research and dialogue, paving the way for a more inclusive and ethical integration of AI in radiology and beyond."
Read the complete paper here and the accompanying editorial here.
