Two recent studies show that natural language processing (NLP) technology can accurately describe trends from different pathology findings in neuroradiology reports and determine additional recommended imaging tests. Both studies were conducted by Partners HealthCare of Boston, with results presented at the 2007 RSNA meeting in Chicago.
One study found that while "accurate determination of recommended imaging tests and the time frame is possible with an NLP engine," most radiology reports with recommendations "are nonspecific for both modality and time frame." With the additional information, imaging centers and other providers can assess the appropriateness of referral practices and radiology self-referral policies, potentially reduce unnecessary scans, and improve their own operations.
The first study reviewed the value of data mining with an NLP program and online analytic processing in classifying pathology findings used in clinical referral trends and research. Specifically, researchers sought to validate the accuracy of both technologies in classifying unstructured electronic neuroradiology reports with relevant neurology findings.
Researchers reviewed 120 neuroradiology reports between 2005 and 2006, selecting reports with and without relevant neurology findings from an unstructured report database. A radiologist and an NLP program then independently categorized the randomized reports on the pathology of the neurology findings.
Statistical tests also were performed to calculate the specificity, sensitivity, accuracy, and positive and negative predictive values of the NLP program in classifying the neuroradiology reports with the relevant findings according to their pathology. In all, researchers assessed 207,377 neuroradiology reports with relevant findings (between 1995 and 2006) to determine the trends for the different pathology findings.
High sensitivity
The analysis found that the NLP technology's ability to classify pathology findings resulted in a sensitivity of 99%, specificity of 75%, accuracy of 96%, positive predictive value of 96%, and negative predictive value of 93%.
Of the 207,377 neuroradiology reports in the study, the most frequently observed pathology finding was vascular at 75%. The category included vascular pathologies, such as hemorrhage (21%, which included acute, subacute, chronic, extravascular, collection, generic hemorrhage), dilation (5%), infarction (28%), ischemia (23%, which included acute and generic), and obstructive process (20%).
Other specific pathologies noted by the technology were atrophy (13%), neoplasms (7%), infection (3%), meningioma (2%), hydrocephalus (1%), microadenoma (1%), and multiple sclerosis (1%).
Thus, researchers concluded that NLP programs can accurately articulate trends of different pathology findings in neuroradiology reports and assist in the collection of relevant research data.
Leximer's capabilities
The second study sought to evaluate recommended imaging procedures and their time frame in assessing the appropriateness of referral practices and radiology self-referral policies.
The natural language processing engine used in the study, dubbed Leximer (Lexicon Mediated Entropy Reduction), was developed at Massachusetts General Hospital, one of Partners HealthCare's two hospitals in Boston. The technology is owned and marketed by speech recognition and document management firm Nuance Communications. The Burlington, MA-based company bought Leximer in its October 2007 acquisition of New York City-based Commissure, an imaging software company that specialized in speech-enabled radiology workflow optimization and data analysis products.
Leximer was used in conjunction with online analytic processing technology to categorize radiology reports with recommendations based on the chosen imaging modality and time frame. The study also reviewed trends for different time frames for recommendations in a report database that spanned 11 years.
From the database, 120 reports with and without recommendations for further action were selected. Two radiologists and the NLP engine independently classified the randomized reports based on the presence or absence of recommendations.
Reports, which contained recommendations, were then categorized according to imaging modality and recommended time frame. The NLP engine also was used to determine the recommended imaging and time frames for further evaluation or follow-up in 5.5 million reports.
Final analysis
The analysis found that for recommended modalities, Leximer demonstrated sensitivity of 100%, specificity of 84%, accuracy of 95%, positive predictive value of 93%, and negative predictive value of 100%. For the time frames, the NLP engine showed sensitivity of 100%, specificity of 86%, accuracy of 96%, positive predictive value of 94%, and negative predictive value of 100%.
The group also noted that the radiologists did not recommend a specific modality in 21% of the cases analyzed, or the time frame in 89% of the cases. In addition, specific time frames were not stated for most recommended CT and MRI exams.
They concluded that, unfortunately, most radiology reports with recommendations "are nonspecific for both modality and time frame," though the NLP engine can accurately determine recommended imaging tests and the time frame for which a scan is needed and a report generated.
By Wayne Forrest
AuntMinnie.com staff writer
January 24, 2008
Related Reading
Billing analysis makes dollars and sense, November 17, 2006
Workflow, communication tools offer new hope for productivity, August 1, 2006
Copyright © 2008 AuntMinnie.com
