It's assumed that implementing structured reports in a radiology department has a number of benefits, including consistency and understandability of content, more complete reports, increased workflow efficiency, and fewer errors. But how valid are these assumptions?
There haven't been many formal analyses published to validate and verify these beliefs. So a study of error rates of radiology reports created from a library of structured reports, presented at last month's Society for Imaging Informatics in Medicine (SIIM) meeting, generated a lot of interest.
The study was conducted by radiologists from Cincinnati Children's Hospital, which has converted to department-wide use of structured reports for more than 95% of exams. Even with this initiative, the department is not free of final reports that contain errors -- in fact, far from it, according to pediatric radiology fellow Dr. C. Matthew Hawkins.
He reported that in a study of almost 650 reports, researchers found some type of error in 37%. The errors tended to be minor, however, with about half being grammatical and minor punctuation mistakes.
In the presentation, Hawkins discussed 644 randomly selected radiology reports prepared after the department had adopted structured reporting, and compared them to 311 randomly selected reports that had been prepared when the department only used standardized report templates that still required dictation of content.
These latter reports had previously been analyzed for errors as part of an earlier study published in the Journal of Digital Imaging. The JDI study compared the error rates and turnaround times of free-form dictated reports versus reports from a standardized report template that contained prepopulated patient and exam-related information (JDI, August 2012, Vol. 25:4, pp. 504-511).
The same two radiologists who evaluated the 311 reports of the prior study also examined the 644 reports. They calculated the average number of errors per report, the average number of nongrammatical errors per report, the percentage of reports with an error, and the percentage of reports with a nongrammatical error.
"While we were disappointed by the number of errors we found in the structured report sample, we were very pleased to verify that the number of nongrammatical errors had decreased by 16% compared to errors identified in reports where standard report templates only were used," Hawkins said.
The reduction in nongrammatical errors was largely due to fewer commission errors (a retained statement from a template that might change the meaning of a report) and missense errors (mistakes that change the meaning of a phrase or sentence). These are the types of errors that can confuse the recipient physician and possibly affect patient management, he explained. Human errors altering the meaning of a phrase or sentence were comparable.
The number of nonsense errors increased, from 0.05 per report with standardized template-only reports to 0.4 with the structured reports. The number of translational errors that did not alter the meaning of a sentence, improper period errors, and omission errors that did not change the meaning of a phrase or sentence were comparable.
The researchers did not analyze the reports by individual radiologist or by level of professional training, such as differentiating the accuracy of radiologists, fellows, or residents as a group. This was beyond the scope of the study and would have required a much larger sample of reports in both categories, as the department is very large, Hawkins explained.
When the department created its structured reports, many different individuals were involved during a process that took 18 months. This produced high-quality "canned" reports, whose content included only the necessary pertinent positive and negative statements required for diagnosis.
When asked if individual radiologists whose structured reports were included in the study were told about their errors, Hawkins said they had not been given the results. However, the department will probably start providing individual feedback to radiologists about errors identified as an ongoing quality initiative to continue to reduce error rates.
"Ultimately, however, we believe that grammatical errors in content added to the structured report by the radiologist preparing it will not be reduced until vendors supply grammatical error cues in their dictation software," he concluded.
