When it comes to predicting success on the American Board of Radiology (ABR) qualifying exam, more emphasis should be on resident performance rather than prior standardized testing results, says an editorial published August 30 in Academic Radiology.
In her editorial, Dr. Desiree Morgan from the University of Alabama at Birmingham wrote that focusing on evaluations of clinical service during residency can allow for radiology educators to better employ interventions to help aspiring radiologists with exam preparation.
"We all work hard to train residents to be effective providers of safe, high-quality radiology care for our patients," Morgan wrote. "Well-trained residents are not characterized by one variable."
The researchers have been long interested in finding ways to predict whether radiology students will be successful when taking their board exams. For educators, prediction tools can help identify students in graduate medical education training programs who are at risk of failing. Morgan wrote that this way, interventional strategies can be developed and implemented early in education.
"These implementation strategies, when successful, benefit the individual as well as the training program, which is bound by the Accreditation Council for Graduate Medical Education [ACGME] core program requirement of an 80% first-time certifying exam pass rate during any three-year look back for residencies in which the certifying board holds annual exams," she added.
While tests taken before the qualifying exam have been explored in predicting success, Morgan pointed to a 2019 study led by Dr. Gary Lloyd Horn from Baylor University that found "significant correlation" between lower subjective faculty evaluation scores and failure of the qualifying exam.
Horn and colleagues found that exam takers with first-year scores below the 30th percentile or third-year scores below the 20th percentile should be concerned about potential failure. However, the score in the second year had no statistical association.
"This heterogeneity points to several issues that limit the strength of using the in-training exam as a predictor. First, resident study patterns for the in-training exam vary widely, and secondly, the emphasis a program might place on individual in-training exam results varies even more," Morgan wrote.
A second study by Horn and colleagues that Morgan cited, published in Academic Radiology in 2022, found that first-attempt failure on the ABR qualifying exam was tied to lower scores of resident performances on rotations. Morgan called this study "a call to action" to look more closely at resident performance.
She wrote that radiology educators are encouraged to look more broadly at resident performance on rotations as documented with milestones. She added that this can help understand if the Horn team's findings are valid on a larger scale.
"These training-level specific achievement rankings or scores mandated by the ACGME have broader applicability to the radiology learning community at large and potentially more power to aid a similar but multi-institutional approach," Morgan wrote.
Morgan also pointed out that while understanding the relationship between ACGME milestones in diagnostic radiology and certification outcomes in multiple institutions will be challenging, using these findings to better identify trainees at risk for failure is worthwhile.
"Our residents at risk deserve nothing less in our continued efforts to improve the learning environment to enable their professional success," she wrote.
