Radiology residents may not always know what changes have been made to their preliminary reports by attending radiologists, potentially missing out on valuable educational opportunities. Software developed by a team from Johns Hopkins Hospital aims to close this communication gap.
The tool captures preliminary reports from residents from RIS data, as well as final reports from attendings, allowing easy tracking of changes to reports. Since adopting the system, it's been used an average of more than 20 times a day, said Dr. Paras Khandheria.
He presented the institution's experience with the application during a scientific session at the recent Society for Imaging Informatics in Medicine (SIIM) 2013 annual meeting in Grapevine, TX.
Feedback models
In radiology residency programs, there are several workflow models typically used to provide residents with feedback on changes made to their preliminary reports by attending radiologists.
In a model based on overnight or autonomous review, a resident independently previews a study and generates a preliminary report. Later, the attending reviews the study, edits it, and finalizes the report. If the attending chooses to, he or she notifies the resident of any changes to the preliminary report, Khandheria said.
In a semiautonomous review model, the attending reviews and discusses the study with the resident after the resident has previewed the study and generated the preliminary report. The resident then edits the report, followed by the attending radiologist editing and finalizing the report.
In a directed review model, the resident first previews the study. After the attending reviews and discusses the study with the resident, the resident then generates the report. In the final step, the attending edits and finalizes the report.
Communication breakdown
Regardless of the workflow model, the attending's final edits are often made without the residents receiving additional feedback, Khandheria said.
"Sometimes those changes may not be that substantial, maybe just for grammatical or stylistic issues, but sometimes they could be true interpretive errors for which feedback might be useful," he said.
While residents always have the option of manually querying the RIS to see what the final reports said, manually polling individual reports for minor changes via the RIS is an inefficient and laborious process.
As a result, minor changes to residents' reports are often not reviewed, even though they may have educational value, Khandheria said. Unfortunately, current RIS technologies lack the functionality to support viewing of change tracking across different versions of the same radiology report.
Capturing all report versions
As a result, the Johns Hopkins team built a tool to capture residents' preliminary reports and attendings' finalized reports from the RIS. A change-tracking utility then compares the reports. The group also developed a clinically integrated front-end Web interface to display the results and enable efficient viewing of workflow, according to Khandheria.
The software tool allows residents to easily track changes made by attending radiologists during finalization of radiology reports. Image courtesy of Dr. Paras Khandheria.The tool supports multiple ways to search for reports, such as by medical record numbers (MRNs), study accession number, date range, etc. It also incorporates role management functionality.
"If you're a resident using the system, you can only visualize the reports that you yourself are an author on," he said. "If you're an attending, you can visualize any reports that any of the residents working under you have authored and have been assigned to you. And, for example, if you're a quality director or a program director or someone like that, you can utilize all the reports across the system."
Users are able to quickly see changes in the final report via the change-tracking utility, he said.
The application is integrated with PACS and the university's desktop context management system, which allows the changing of context for retrieval of study images. The context management system also aggregates clinical data from the institution's electronic medical record (EMR), such as prior radiology studies, clinical notes, and laboratory values, according to the researchers.
Users can also toggle between different versions of the report.
System architecture
Radiology messages from the hospital IT system are currently stored in a data warehouse operated by Analytical Informatics of Baltimore. A proprietary software development kit (SDK) allows this dataset to be queried for the different versions of the radiology reports, Khandheria said.
The Johns Hopkins tool is built with the JRuby on Rails programming language, which fetches information from the data warehouse. The front end is built with HTML, Cascading Style Sheets (CSS), and JavaScript (jQuery framework).
Over a six-month period from November 24, 2012, to May 24, 2013, there were 3,921 visits to the system, representing approximately 22 visits per day. There were 84,659 total actions, such as searching and opening a report, for approximately 22 actions per visit. The average visit duration was five minutes and 13 seconds, Khandheria said.
As might be expected, the first-year radiology residents used the system the most, he said.
In the future, the researchers are considering mining data to assess resident performance, normalized perhaps by modality, attending, and resident year.
"So you could see, as a resident compared to other residents who are here working with the same attending and reading the same kind of report, how many changes are being made to your report versus your colleagues," he said.
The researchers are also considering implementing a more sophisticated change-scoring system, perhaps by using additional flags, weighting systems, or natural language processing. In addition, they would also like to integrate feedback systems directly into the worklist, such as allowing attendings to flag cases for resident review with reading receipt confirmation, Khandheria said.
