A study of radiology residents found that chest x-ray diagnostic accuracy significantly declined after overnight shifts, and AI assistance could not restore performance to pre-shift levels or meaningfully improve accuracy in fatigued readers, raising concerns about patient safety and the adequacy of current duty-hour regulations.
- Diagnostic accuracy in fatigued radiology residents dropped from 0.71 to 0.61 after night shifts, with AI providing no significant improvement.
- Six of 10 residents showed measurable fatigue signs through eye-tracking technology, while all residents experienced some accuracy decline post-shift.
- AI assistance failed to restore pre-shift performance levels and did not significantly improve diagnostic accuracy in either fatigued or non-fatigued readers.
- Reporting speed remained stable after night shifts, but fixation duration and saccade count decreased, indicating reduced visual attention to x-rays.
- Researchers suggest more comprehensive AI models incorporating confidence scores or broader finding detection may be needed to support fatigued radiologists.
Chest x-ray reading accuracy declines significantly in fatigued radiology residents after overnight shifts, and AI offers little help, according to a recent study.
A team from Aachen University Hospital in Germany measured speed and accuracy among 10 residents when reading x-rays before and after a night-shift, and with and without AI, with findings suggesting AI may not be the urgently needed solution to mitigate radiologist fatigue.
"Night-shift-induced fatigue impaired diagnostic accuracy without slowing reporting speed, raising patient safety concerns. The utilized AI model did not restore pre-shift performance," the researchers noted. The study was published October 22 in European Radiology.
Rising study volumes and persistent workforce shortages are intensifying radiologist workload, the authors wrote. Prior research has linked fatigue to decreased detection of clinically relevant findings and reduced diagnostic confidence, they added. Although AI tools have shown potential to improve accuracy and alleviate workload, evidence that they can offset fatigue-related performance loss remains limited, the group noted.
To explore the issue, the group recruited 10 radiology residents (mean experience, 35.5 months) who each interpreted three sets of 33 bedside chest x-rays in a prospective intra-individual reader study, completing two sessions before a first-call on-site night shift and one session immediately after, the latter with and without AI assistance. They used an AI model they previously developed at their institution and which had shown excellent agreement with expert radiologists.
Technical setup and customized DICOM viewer. a) Radiology resident reads chest x-rays on a single-screen workstation equipped with screen-based eye tracking (bar unit below the screen). b) The x-ray (right) was displayed alongside the structured reporting template (left). c) Without AI assistance, readers reported the presence and severity of findings using drop-down menus. d) With AI assistance, the template was pre-filled with the model’s suggestions, which could be accepted or modified.European Radiology
Screen-based eye tracking identified fatigued readers by measuring how efficiently their eyes moved across each x-ray (saccadic main sequence slope), while diagnostic accuracy was gauged by how closely each resident’s interpretations of five findings (cardiomegaly, pulmonary congestion, pleural effusion, pulmonary opacities, and atelectasis) matched those of a panel of expert radiologists.
According to the analysis, six of 10 residents were classified as fatigued. Their diagnostic accuracy declined significantly post-shift, from a κ of 0.71 to 0.61 (p < 0.001), with concurrent reductions in fixation duration (p = 0.010) and saccade count (p = 0.015). Non-fatigued readers showed a smaller but still significant accuracy decline (κ, 0.72 to 0.65; p = 0.037). Reporting times held stable in both groups (p ≥ 0.463). AI assistance did not significantly improve accuracy in either group (p ≥ 0.525), and shortened reporting times significantly only in non-fatigued readers (by 9.9 seconds; p = 0.031).
"Assistance by the specific AI model used in this study did not lead to a significant improvement in diagnostic accuracy and failed to restore performance to pre-shift levels, suggesting a limited ability to offset fatigue-related declines in performance," the authors wrote.
Ultimately, the findings raise questions about whether current duty-hour regulations and staffing models adequately address fatigue-related performance risk, the authors noted. Future work should evaluate whether more comprehensive AI tools, such as systems that incorporate confidence scores or broader finding detection, can better support fatigued readers, they suggested.
"More comprehensive or differently integrated AI models may provide different levels of support in future studies," the authors concluded.
The full study is available here.
