ORLANDO - Speech recognition systems generate significantly higher rates of transcription error when used by women when compared to men, according to new research presented Monday at the annual meeting of the American Roentgen Ray Society (ARRS).
Dr. Syed Ali from the University of Maryland School of Medicine in Baltimore presented results from a multicenter trial that compared error rates between male and female radiologists using speech recognition technology.
The study was prompted by the recent opening of the school's new state-of-the-art reading room, which utilizes speech recognition for dictation. The room includes a number of features to minimize background noise, such as wall-mounted sound dampeners.
Despite the technology, the facility noted anecdotal evidence from its own residents and attending physicians that female readers were experiencing more errors in reports generated with speech recognition than male colleagues. The group decided to look into the phenomenon further by researching the factors that can affect the accuracy of speech recognition technology.
The protocol
Five male and five female radiology residents were native English speakers and were trained on a commercial speech recognition application. The study standardized the reading environment for background noise and ambient conditions. The reports covered all imaging modalities and included multiple organ systems.
"The residents were instructed to read through the reports and not to correct any mistakes that the recognition software made," Ali said. "They also were told to simulate normal dictation patterns to prevent speed reading and possible false positives."
Each resident dictated a standardized set of 10 radiology reports, containing a total of 2,123 words.
Using commercial software (RadWhere version 8.1, Commissure, New York City), the generated reports were compared to the original reports, and error rates were calculated. "The error rate was defined as the sum of the number of word insertions and deletions divided by the total word count for a given report," Ali said.
When all was said and done, the results found a significantly higher rate of speech recognition error among female residents compared to male residents. The error rates in the male population ranged from 0.025 to 0.139 with a mean of 0.082 (standard deviation = 0.033). Error rates in the female population ranged from 0.015 to 0.205, with a mean of 0.100 (standard deviation = 0.044).
"There was some variability between men and women with some overlapping performance," Ali noted. "Interestingly, the lowest error of 1.5% and the highest error rate of 20.6% on a single report were (read) by female residents."
Potential bias
Among the potentially confounding factors in the study was a potential bias from the level of residents' training. The average male resident's training was slightly higher than the female training, but everyone was trained equally on the speech recognition software.
Ali also cautioned of a potential discrepancy due to volume and pitch of voice between the genders, as well the speaking style among the readers, if the residents read faster as the reports continued.
Other potential influences on the study's outcome included computer processor power, microphone quality, and background noise.
"Additional work should be done to determine if women can be trained to increase their accuracy rates," Ali said. "Speech engine vendors also should refine their programs to improve accuracy."
Further work also should investigate the causes of these errors with larger sample sizes, he said.
Also participating in the study were co-authors from the Hospital of the University of Pennsylvania in Philadelphia, Johns Hopkins University in Baltimore, and the VA Maryland Health Care System in Baltimore.
By Wayne Forrest
AuntMinnie.com staff writer
May 8, 2007
Related Reading
Speech recognition cuts report turnaround times, May 4, 2007
RIS-SR-PACS combo speeds radiology reporting, April 17, 2007
Speech recognition technology shows double-digit error rate, November 30, 2006
HIT needs, cost factors drive speech recognition implementation, September 27, 2006
Meeting the challenge of structured reporting, June 23, 2006
Copyright © 2007 AuntMinnie.com
