WASHINGTON, DC - There may be no better way to evaluate the competency of budding radiology residents than to expose them to the actual hands-on, frenetic experience of the clinical environment. A simulation program that replicates the reading room was described this week at the American College of Radiology (ACR) annual meeting.
Researchers at Southern Illinois University (SIU) developed a novel radiology simulation program that uses specific events and challenges from a hospital shift to evaluate a radiologist's diagnostic, communication, professional, and service skills. With each task, residents are evaluated on their ability to complete their duties, including interpreting images, interacting with human caregivers and patient mannequins, and even handling computer glitches.
"Complete immersion simulation provides the opportunity to train and evaluate many aspects of being a radiologist," said lead author Dr. Andrew Sherrick, chairman of the radiology department at the SIU School of Medicine in Springfield, IL. "Each event of the simulation is designed to evaluate a specific quality, and scores are defined for each category."
Real-world training
While simulation training can provide a safe bridge between the classroom and real-life clinical experience in any medical discipline, the technology is most applicable to radiology, Sherrick said.
Dr. Andrew Sherrick from Southern Illinois University."Most patient encounters in the medical field are difficult to completely reproduce in a simulation environment and require the use of mannequins or actors and actresses," he said. "For example, the encounter of a primary care physician or surgeon with a patient can be very difficult to exactly reproduce with simulation. On the other hand, the patient encounters in diagnostic radiology are different from patient encounters in other specialties."
With radiology, simulation can "match the real-life experience down to every detail, including the exact image, the exact history, and the exact timing the exam populates the worklist," Sherrick told the ACR 2017 attendees.
The SIU radiology simulation lab is housed at the Memorial Center for Learning and Innovation (MCLI), which is part of the Memorial Health System. The simulation program places radiology residents in the position of an on-call physician for an after-hours shift at a high-volume community hospital with inpatient services, a busy emergency department, a level I trauma center, and a stroke center.
The room looks like any typical radiology reading room with a PACS, dictation system, and phone. It is also equipped with an audiovisual monitoring system to review the subject's performance and provide feedback at a later time. A simulated CT room is next door, where a mannequin can mimic patient symptoms and radiologic emergencies such as adverse contrast reactions.
The program simulates a radiologist's workload and work environment as close to real life as possible. Imaging exams from all modalities populate the workstation at predefined intervals to match a typical workday. To avoid selection bias in the simulated scenarios, Sherrick chooses a random shift at SIU from the previous month, taking cases from a 24-hour period of that day.
"A unique feature of the program is that the timing is tightly choreographed down to the minute to reflect the workload of an actual shift," he said. "We want to see how you perform in a real-world environment. It is not only a matter of whether you can diagnose active bleeding from an artery in a trauma patient; it also matters that you get that report out in a timely fashion. If you are 45 minutes late getting that report out, it may be too late, even if you get the diagnosis correct."
The program also inserts common distractions a radiologist might encounter, such as phone calls and visits from doctors, nurses, and technologists, during the course of a shift. At some point during the scenario, a computer glitch occurs to see how the subject would react.
"The reason for this is to mimic an actual work environment and create scenarios that can be used for additional teaching and evaluation of how you manage those scenarios," Sherrick said.
Performance criteria
As the radiologist performs the mock tasks, he or she receives performance scores based on four major criteria:
- Medical diagnostic skills: The ability to detect abnormalities, the person's medical knowledge, the ability to integrate information from different sources, and medical management
- Communication skills: The clarity of radiology reports, editing errors, and how the person communicates critical results and performs appropriate documentation
- Professionalism: Team management skills, leadership, and compassion
- Service: When the person starts and ends his or her shift, completion of work, report turnaround time, team effort, and willingness to do more when needed
For example, a participant who starts his or her shift early gets five points, while one who is reading a case when the shift begins gets two points. Someone who arrives on time but dawdles, gets a cup of coffee, and logs in 10 minutes late receives no points in the service category.
Sherrick provided a sample simulation scenario during his ACR 2017 presentation. A radiologist begins with two imaging exams to interpret. Two minutes later, an ankle x-ray pops up, followed five minutes later by a CT scan of a patient's head, chest, and pelvis. At the 11-minute mark, a technologist calls with a question about whether to give a patient contrast for a specific CT scan.
"All the while you are reading these exams and interacting with the technologist, you are accumulating scores in the four specific categories," he explained.
As the program continues, a CT of the abdomen and pelvis appears for a patient with right lower quadrant pain.
"In this case, we are not only looking at your diagnostic skills, but also assessing your report for clarity and editing errors," he added. "You also earn points for communication skills."
Next, a technologist enters the reading room to tell you a patient is having trouble breathing after a CT scan. The radiologist goes next door to the simulated CT room where the mannequin appears to have an adverse contrast reaction. There is a human nurse in the room as well.
"Hopefully, you will assess and treat the reaction and receive points for medical management, team management, leadership, and communication," Sherrick said. "While you are dealing with that contrast reaction, your PACS continues to populate cases. How quickly you deal with the contrast reaction will determine how many cases you have when you get back to the worklist. This all ultimately affects your turnaround time, so you have to be efficient. You don't want to take 30 minutes to deal with the contrast reaction when you can deal with it in 10 [minutes]."
Extended applications
So far, 20 residents have participated in the program. Sherrick conceded he has not tried the program himself just yet, but he sees the technology as having practical applications beyond its current objectives.
"I see this not only being used as a tool for resident training and evaluation, but as a tool for per-employment assessment as an objective measure, if you were looking to hire a radiologist in your practice," he said.
The cost to establish the simulation lab was minimal for SIU because Memorial Health System was looking for a tenant for the recently completed innovation center. Sherrick speculated that creating a simulation lab from the ground up would involve considerable costs. Still, he strongly believes in the benefits of such a technological venture.
"It creates great opportunities to talk to residents about things that we may not normally talk to them about," he said. "If a resident shows up five minutes late [in the real world], we might let it go. But in this situation, they have to show on time. It is important that they know they have things to work on."
