Radiologists prefer computer-aided detection (CAD) software that gives them necessary information with the fewest clicks possible and is closely integrated with PACS, rather than applications that require multiple manual steps to interrogate data, according to a team of German researchers.
In an article published online March 26 in the Journal of Digital Imaging, the team from RWTH Aachen University reported that radiologists clearly preferred a more automated CAD/PACS integration approach that yielded efficient handling of image data and fast delivery of results over methods that required manual and intermediate steps to utilize CAD functionality.
"Seamlessly integrating CAD into the PACS without additional work steps or unnecessary interrupts and without visualizing intermediate images may considerably improve software performance and user acceptance with efforts in time," wrote corresponding author Thomas Deserno, PhD, head of the university's image and data management division, and colleagues.
For a number of reasons, image processing in general is not widely used in routine clinical practice. First, the algorithms have not been evaluated comprehensively on daily image use, Deserno told AuntMinnie.com.
"Only a few images are used, but image processing and CAD are not robust enough to perform on a regular daily basis," he said.
In addition, CAD techniques so far have been based on analysis of images only. In contrast, the radiologist's diagnostic task is based on the overall patient case, Deserno said.
Furthermore, CAD is usually designed as a standalone technology, ignoring the workflow of the physicians and radiologists. The researchers sought to address this issue in their study, focusing on an example of BoneXpert (Visiana), a software package that determines skeletal age based on a radiograph of the left hand.
Different integration models
The researchers evaluated the various usability aspects of two different levels of CAD/PACS integration that were provided by its developer. In the first method, a plug-in was used to load images manually into the application via a temporary folder on the hard drive. The second method, BoneXpert Intray, relies on the DICOM protocol to feed an "intray" of images for analysis. Manual steps by the physicians are required in both methods.
The researchers surveyed seven radiologists from the residency program at University Hospital Aachen on the two methods using "think-aloud" audio records and a questionnaire. The radiologists, who had pediatric radiology experience ranging from one to four years, examined 12 to 14 radiographs during the "think-aloud" session. Each method's integration of data, function, presentation, and context were analyzed.
For both integration variants, the CAD module was easily integrated into workflow, and it was judged to be comprehensible and to fit in the conceptual framework of the radiologists. Weak points, however, were observed in terms of data and context integration, according to the authors. Overall, radiologists slightly preferred the Intray module.
A surprising finding was that visualization of intermediate image processing states was less important to the radiologists than efficient handling and fast computation. Several times during test sessions, participating radiologists expressed a wish for BoneXpert to be able to directly analyze the bone age of the selected hand radiograph without the need to select and load files from folders or lists and repeatedly click the "analyze" button, as required in both integration methods, according to the researchers.
Seamless integration
This problem would be solved by an integration version that just provides the BoneXpert results window. It would also avoid the problems encountered in the study with the file directory tree and the data sheet of DICOM files, according to the authors.
"According to this final finding, a third integration variant of BoneXpert has been developed in the meantime, allowing the automatic processing [of] radiographs in batch mode without displaying the intermediate processing result," the researchers wrote. "This will foster applicability and acceptance for clinical routine."
Image processing software developers have frequently tried to mimic physicians with their algorithms, and they have implemented software that can be "understood" by the physicians, Deserno said.
"Statistical methods, for example, such as simple number crunching, which might perform well in CAD evaluation, were said to be [not suitable for integration] because the user (physician) cannot explain the results obtained," according to Deserno.
However, this study showed that the radiologists do not like to have the CAD results explained to them, at least if this step requires further system interaction, he said. The easier and faster the computation process, the better the system.
"In other words, the old paradigm that image processing in medicine shall mimic the physician and needs to transparently deliver its computation was disapproved," he said. "This may also be one reason why the BoneXpert software is not yet in broad use, although this software works sufficiently robustly on unselected daily data."
Broad applicability
The research results would be expected to apply to other CAD applications as well, Deserno said.
"Consider the workflow of the radiologist and the required interfaces when trying to establish image processing in diagnostic routine," he said.
For more widespread use of CAD to occur, large databases with reliable ground truth are needed for comprehensive evaluation of CAD algorithms. In addition, "smart" methods should be developed that can process multiple images from multiple imaging devices simultaneously rather than sequentially.
"CAD will be successfully applied in the clinics when it can cope with a complete case composed of images from different modalities and points of time rather than a single diagnostic image," he said.
