Monday, November 27 | 1:50 p.m. - 2:00 p.m. | M6-SSNPM01-3 | E351
Researchers from the Yale School of Medicine have developed a patient-centric radiology literacy tool using natural language processing (NLP) technology and prompt engineering to produce radiology reports that are more readable for patients.
Patient communication can be a perennial problem in radiology because while the American Medical Association recommends health information be relayed to patients at a third- to seventh-grade reading level, research has shown that the average radiology report is written at the 13th-grade level, according to presenter Rushabh Doshi. Because radiology reports frequently feature intricate medical language that can lead to confusion, distress, and incorrect interpretations for patients, a radiology report literacy tool could offer a patient-friendly interface that can be distinguished from existing resources.
While further investigation is needed to determine the accuracy of the outputs in clinical practice, the proof-of-concept study used OpenAI’s application programming interface (API) for critical tokens within the input sequence and the Delphi method to select basic and useful prompts. Doshi will discuss how the engineered prompts were tested and performed on 25 radiology reports from the MIMIC-III database.
A total of 110 prompts were assessed using a stepwise process of modifying prompts based on context and grade level, with the top five prompts tested on 209 radiology reports to select the best prompt for the interface. Readability was compared using Gunning Fog, Flesch-Kincaid Grade Level, Automated Readability Index, and Coleman-Liau indexes. A paired Wilcoxon Signed Rank test compared readability scores.
According to Doshi, simplified reports generated through the tool “demonstrated a statistically significant enhancement in readability compared to the original radiologist reports.”
The work highlights the significance of prompt engineering in generating optimal outputs from NLP models, according to the researchers. Learn more at this Monday session.
