Incorporating point-of-care clinical decision support (CDS) into a computerized physician order-entry (CPOE) system can yield a modest improvement in the appropriateness of advanced imaging orders in the inpatient setting, according to a study published online January 23 in the Journal of the American College of Radiology.
The team from Henry Ford Health System, led by Dr. Andrew Moriarity, found that CDS yielded a small but statistically significant improvement in the appropriateness of advanced imaging orders following adoption of software based on American College of Radiology (ACR) Appropriateness Criteria (AC). Primary care physicians improved the most, the group noted.
Effective in the outpatient setting
Previous research has shown that CDS can be effective in the outpatient setting for reducing inappropriate imaging as well as overall imaging volume. So the Henry Ford team sought to focus specifically on the previously unstudied effect of CDS on the appropriateness of inpatient imaging, Moriarity told AuntMinnie.com.
Dr. Andrew Moriarity from Henry Ford Hospital.Over a 12-month period from October 1, 2011, through September 30, 2012, the researchers assessed how providers at the health system's 800-bed level I trauma center and a 200-bed community hospital used the CDS pull-down menus in the CPOE system to order nuclear medicine, CT, and MR studies in adult patients. Scores were also generated to measure the extent to which their orders corresponded with the ACR criteria (JACR, January 23, 2015).
The pull-down menus for "clinical scenario" and "patient signs and symptoms" generated structured data to be used by CDS and were directly populated from the ACR AC guidance documents, according to the researchers. While ordering physicians could also utilize free-text fields to provide patient history and comments, such entries could not be processed by the CDS system due to their unstructured format.
In the initial six months, orders were processed just via CDS pull-down menus to provide a baseline period for the study. Over the next six months, after placing their imaging order in the same fashion, ordering physicians were also provided with the ACR AC scores for their orders, and they were asked to acknowledge the score before finalizing their request. This was considered to be the intervention period of the study.
Scores were rendered on a scale of 1 to 9. Orders with scores from 1 to 3 were considered "inappropriate," while 4 to 6 indicated an "uncertain" order and 7 to 9 meant an order was "appropriate."
Effect of viewing AC scores
Overall, there was no statistically significant difference between the two study periods in terms of the number of advanced inpatient imaging orders. Notably, most requests did not generate an ACR Appropriateness Criteria score, due to missing information in either the "clinical scenario" or the "patient signs and symptoms" structured data fields.
"We were surprised at the large percentage, approximately three-fourths, of the requests that could not be matched to existing appropriateness criteria," Moriarity said.
On the positive side, this percentage dropped from 73.1% to 68.9% in the second half of the study, a statistically significant improvement (p < 0.001).
"An additional 0.9% of baseline-period requests and 0.8% of intervention-period requests were not scored because the requested modality was not present in the matching ACR AC guidance document; this difference was not significant," the authors wrote.
| Effect of CDS on appropriateness scores | ||
| AC score categories | Baseline period | Intervention period |
| Appropriate | 76% | 81.7% |
| Uncertain | 19.2% | 14.2% |
| Inappropriate | 4.8% | 4.1% |
While the increase in orders considered to be appropriate was statistically significant (p < 0.001), the decline in orders categorized as uncertain or inappropriate was not. The researchers also observed a slight but also statistically significant (p < 0.001) increase in the average AC score, from 7.2 ±1.6 to 7.4 ± 1.5.
In other findings, the researchers detected a significant increase (p < 0.001) in the average appropriateness score for nuclear medicine requests (from 6.6 ± 1.9 to 7.3 ± 1.6), as well as a slight but significant decrease (p < 0.001) in the percentage of requests that lacked sufficient structured data to automatically generate CDS (from 73.4% to 71.7%).
However, the small improvement in average CT score (7.2 ± 1.8 to 7.3 ± 1.6) did not reach statistical significance. Furthermore, the average MR appropriateness score did not change (7.4 ± 1.4 for both periods). There was a slight but statistically significant (p < 0.001) decrease in the percentage of CT requests that lacked sufficient structured data to automatically generate a CDS score (from 73.4% to 71.7%).
Primary care docs benefit the most
While both primary care providers and specialists had higher average ACR AC scores during the intervention period, the primary care providers had the biggest improvement, climbing from 6.9 ± 1.9 to 7.4 ± 1.6. Specialists improved from an average AC score of 7.3 ± 1.6 to 7.4 ± 1.5. Both improvements were statistically significant (p < 0.001).
"The higher AC scores for specialists in both periods may indicate that advanced imaging use by specialists more closely followed the ACR AC guidelines compared with use by primary care providers," the authors wrote.
The study team also found that physicians who had graduated in the most recent decade had the highest baseline AC scores. With CDS, however, all other age cohorts were able to meet or exceed those initial scores, according to the authors.
"Physicians graduating before 1980 showed the greatest improvement in AC scores during the intervention period and were the only group to show an improvement in the percentage of requests lacking sufficient clinical information to allow CDS to generate an AC score," they wrote.
The use of natural language processing techniques to extract data from free-text information to make it usable by clinical decision-support software should be explored as a way to improve the appropriateness of imaging orders or reduce the amount of them, according to the researchers. Expanding the available guidelines should also help improve the number of clinical scenarios with appropriateness recommendations, Moriarity said.
The authors also "encourage referring providers to directly consult radiologists when tests are considered of uncertain benefit or inappropriate in order to provide the right examination for each patient," he said.
