A new study found that both radiologists and large language models perform poorly at distinguishing AI-generated chest x-rays from real ones, with neither group reliably accurate enough to serve as a primary safeguard. The researchers recommend treating image authenticity as a data governance problem through cryptographic credentials or watermarking rather than relying on visual inspection alone.
- Radiologists and LLMs showed similar overall accuracy but failed in different ways: radiologists better identified diffusion-model images (82-100%) while LLMs had higher specificity for real images (90-100%)
- Top performers were Gemini 3.5 Flash (77.9% balanced accuracy) and one radiologist (75.8%), with no statistically significant difference between them
- Poor agreement across all evaluators (Fleiss' kappa: 0.213) indicates visual detection alone cannot reliably distinguish synthetic from authentic x-rays
- Risk occurs when AI-generated images enter training datasets mislabeled as real, potentially distorting model output and weakening imaging value
- Recommended solution: implement cryptographic content credentials or watermarking at image creation and transfer, with visual review as secondary control
Improved strategies may be required to sniff out AI-generated chest x-rays in medical datasets, according to research published October 13 in the Journal of Imaging Informatics in Medicine.
The finding is from a study that tested radiologists and large language models (LLMs) on 40 real and 120 synthetic chest x-ray images, with radiologists more often identifying synthetic images that LLMs classified as real, and LLMs more often classifying real images that radiologists identified as synthetic, noted a team from Seoul National University Hospital in South Korea.
“The central finding is therefore not which evaluator performed best but how each failed,” the group wrote.
Synthetic chest x-rays generated by AI are increasingly used to supplement training datasets, according to the researchers. Yet a risk arises when generated images enter a training corpus, benchmark, or clinical archive mislabeled as real, which can distort output and weaken the value of imaging, they noted. Whether experienced human readers or LLMs can reliably distinguish synthetic from authentic x-rays had not been established, they noted.
To explore the issue, the researchers tested two radiologists (RAD1, 12 years post-MD with seven years of dedicated thoracic radiology; RAD2, 13 years post-MD, nonthoracic subspecialty), two general practitioners, and three LLMs (Gemini 3.5 Flash, GPT-4o, and Claude 3.7 Sonnet) on classifying 160 chest x-rays: 40 real and 120 synthetic images generated by three different AI pipelines. Balanced accuracy was the primary metric, given 75% synthetic prevalence, the researchers noted.
According to the results, the top performers were Gemini 3.5 Flash (balanced accuracy: 77.9%) and RAD1 (75.8%), with no statistically significant difference between them. The general practitioners performed near chance. Human readers more accurately identified diffusion model-generated images (82% to 100%) than LLMs (32% to 72%), while LLMs showed higher specificity for real images (90% to 100%) compared with human readers (50% to 100%). Agreement across all evaluators was poor (Fleiss’ kappa: 0.213).
"This study found that the best-performing human and LLM evaluators achieved similar point estimates of balanced accuracy but showed distinct, dataset-specific operating points and error profiles," the authors wrote.
Ultimately, neither humans or LLMs appear well suited to serve as the primary safeguard in this issue, according to the researchers. The observation points toward treating provenance as a data governance problem addressed at the point of image creation and transfer, through cryptographic content credentials or watermarking, with visual review as a secondary control, they suggested.
"Future studies should test whether visual assessments combined with nonvisual provenance signals can improve verification performance," the authors concluded.
The full study is available here.
