Physicians at Massachusetts General Hospital (MGH) have been using a clinical decision-support system for years to order radiology procedures. They're hoping to make their system even more precise with a new data-mining tool that facilitates analyzing patient symptoms with exam findings.
Attendees at the Society for Imaging Informatics in Medicine (SIIM) annual meeting learned details of the first formal validation exercise of the tool, called RadLex Enhanced Automated Leximer Mining (REALM). Dr. Supriya Gupta, a research fellow in imaging informatics, explained how she and her colleagues applied REALM to identify the most common clinical indications of a patient with the most common findings when a head and neck CT and/or MRI exam was ordered.
REALM is software that uses RadLex-based ontology trees to assess patterns in unstructured radiology reports in conjunction with a natural language processing (NLP) algorithm to map patient characteristics with imaging findings in radiology reports. Its developers hope that by mining large quantities of practice data generated from exams ordered through the clinical decision-support system, the appropriateness of ordering expensive diagnostic imaging procedures may be better quantified.
The MGH research team is in the process of validating REALM. For its first formal validation test, the group selected the clinical indication of the term "subarachnoid hemorrhage" to evaluate CT and MRI head exams. REALM was used to query the RIS database of the hospital's radiology reports; more than 1,000 reports with subarachnoid hemorrhage were manually reviewed to determine REALM's accuracy in matching the actual reports.
Two blinded radiologists manually reviewed the 1,013 reports to evaluate the presence of subarachnoid hemorrhage. They determined that REALM had correctly identified subarachnoid hemorrhage or a related term in an average of 972 cases, or 96%. It was 97.2% accurate, with a sensitivity of more than 99% and a specificity of more than 90%. The positive predictive value of REALM was 99.89%, and the negative predictive value was 63.6%.
 |
| How REALM is intended to interact with both ordering physicians and radiologists. Image courtesy of MGH. |
The researchers then used REALM to retrieve clinical indications selected by referring physicians for ordering CT and MRI of the head and neck between 2000 and 2010 from the hospital's RIS database of almost 6 million reports. These reports were divided into two categories: exams with imaging findings in the report, and exams with no imaging findings in the report. The researchers then queried REALM to identify the type of imaging findings in CT head and neck reports, based on the clinical indication for the exam when it was ordered.
For reasons that could not be explained by the analysis, more men had CT head and neck exams than women, whereas the opposite was true for MRI head and neck exams, Gupta said.
In all, 326 clinical indications were referenced within 28,744 cases of head and neck CT exams ordered to confirm or rule out subarachnoid hemorrhage. The most common reason for ordering a CT head and neck exam was for facial pain (9.15%), followed by headache (7.98%), subdural hemorrhage (5.25%), sinusitis (4.33%), and dizziness (3.99%).
For MRI of the head and neck, 462 clinical indications were referenced out of 58,517 cases. Common indications included non-central nervous system (CNS) primary neoplasm (8.1%), dizziness (6.1%), headache (5.48%), and weakness in either side of the body (3.99%).
Using the most frequently cited indications as examples, Gupta said when facial pain was the indication for a head and neck CT exam, the top three most frequently cited findings in reports were cyst (8.95%), sinusitis (6.03%), and ankylosing spondylitis (5.19%). When non-CNS primary neoplasm was the indication for an MRI exam, the most common finding was the presence of a mass (12.62%), followed by a nodule (9.05%) and metastasis (6.83%).
"The most frequently selected clinical indications of a patient and the most frequently identified positive findings can be used to determine the diagnostic yield of an exam," Gupta said. "They add to the information we already have from evidence-based guidelines about the appropriateness of ordering a given examination."
Co-researcher Thomas Schultz, chief engineer of enterprise medical imaging at MGH, said that having REALM as a screening tool to assess the prevalence of report findings could provide additional information that may be very valuable when medical literature is unclear about how to test certain clinical indications.
"When a patient is experiencing fatigue or has nonspecific symptoms, how do we evaluate them?" he asked. "Our hope is that REALM research will enable physicians to review the historic findings of patients with similar symptoms based on a specific diagnostic imaging exam or several different exams. It will augment clinical decision-support guidelines."
Gupta added that the information could also provide academicians with a sense of trends in disease patterns in patient populations. It also could be used to evaluate the effects of various screening programs.
"Data from a REALM analysis has the potential to assess the appropriateness of performing a specific exam for a specific indication," she said. "It may provide information to improve existing evidence-based guidelines. It may be able to assist a radiologist in doubt."
The research team is just beginning the process of rigorously evaluating REALM. It won't be available for clinical decision support for several months, but the researchers are pleased with the initial tests.
"Our clinical decision-support system is a great tool and has proved very effective," she concluded. "We think that by adding REALM's capabilities, it will become even better."
