Breast imagers fare better with hands-on approach to screening ultrasound

Dec 16, 2004

Even experienced breast imagers can benefit from feedback when it comes to reading breast ultrasound exams, according to the latest data from the American College of Radiology Imaging Network (ACRIN) 6666 trial.

In addition, radiologists who perform breast ultrasound themselves -- versus relying on technologists or others to do the exam -- have better interpretive skills, said the trial's lead investigator Dr. Wendie Berg, Ph.D., during a presentation at the 2004 RSNA meeting in Chicago.

"The ACRIN protocol 6666 is the study of screening ultrasound in high-risk women. The primary aim of the study is to evaluate the performance of whole-breast screening ultrasound combined with mammography and compared to mammography alone," explained Berg, who is with ACRIN in Lutherville, MD. "One of the issues that we really focused on was to establish standards for interpretation and performance, and develop qualification tasks for potential investigators."

ACRIN 6666 began enrolling women in March 2004. The plan is to include 2,808 women at 20 U.S. institutions to determine the exact role of ultrasound in those who are at high risk for breast cancer. The women will receive an annual mammogram and physician-performed sonographic exam each year, for three years.

Berg shared the results of training sessions, in which 35 breast imagers -- each considered an expert with at least two years of experience -- were tested on their interpretive skills of mammographic masses and calcifications, and were also quizzed on ultrasound cases. The quiz results were validated against another potential 35 investigators, Berg said.

The potential investigators, from a cross-section of academic and community practices in the U.S. and Canada, were trained with a one-hour didactic session on the BI-RADS system for breast ultrasound interpretation (BI-RADS US). The readers were also given the specific criteria within the ACRIN 6666 protocol for classifying lesions as negative, benign, probably benign, suspicious, and highly suggestive for malignancy, Berg said. Details on the criteria can be found at the ACRIN Web site.

"For the ultrasound interpretive quiz skills set, there were 70 proven cases with orthogonal views of each lesion," Berg said. "Initially, the investigators were shown the case without any commentary, asked to describe features, and to give their final assessment BI-RADS category."

"Then the second time through studying the cases, they were given feedback after they made their interpretation with the pathology or follow-up results on that case. Then we compared the results with feedback to those without feedback," she added.

Demographic data was collected on the 35 participants who provided the following information:

Number of years doing breast imaging
Percentage of time spent in breast imaging
Who performs the breast ultrasound at the facility?
Number of mammograms read per week
Number of breast ultrasound exams performed and interpreted per week
Current use of whole-breast ultrasound in the practice

According to the overall results, readers did better in ultrasound interpretation when they were given feedback after reading the cases (overall area under the ROC curve, or A_z, improved from 0.81 to 0.84; p < 0.05). Performance also improved among veteran breast imagers, peaking at six to 10 years of experience (A_z = 0.86 after feedback).

"If we look at years in breast imaging, as you might expect, the longer they've been doing breast imaging, the better their performance -- until we get to six to 10 years, and there was actually a little bit worse performance for those who have practiced over 10 years," Berg noted.

While there was no specific reason for this trend, Berg said there was an overlap between radiologists who had practiced for more than a decade and those who relied solely on their technologists to perform the ultrasound scans.

In fact, even with feedback, the group who based their interpretation on a technologist scan still did not perform well, Berg said.

"If we looked at who performs breast US in their practice, those for whom the technologist does their ultrasound had statistically significant lower interpretive skills," she said. In this instance, the A_z was 0.68 without feedback and 0.74 with feedback.

However, interpretive performance was substantially better if the radiologist performed sonography themselves or followed up on the exam after it was performed by a resident, fellow, or technologist (A_z of 0.79 to 0.86 without feedback and 0.81 to 0.88 with feedback).

Looking at the percentage of time spent in clinical breast imaging, those who devoted less than 25% of their average week doing clinical breast imaging had slightly reduced performance (A_z = 0.79) compared to those who spent more than 25% of their time (A_z = 0.84).

In terms of the volume of cases interpreted, Berg said they found a weak correlation with the number of mammograms interpreted per week and performance. Still, readers who interpreted at least 40-50 breast ultrasounds per week did better than those who interpreted fewer than 20 per week. Finally, Berg's group found no observable trends as to how breast ultrasound is currently performed at each investigator's institution.

"Feedback training was a benefit to all observers regardless of how experienced or expert they were," Berg concluded. "We saw improved performance in interpreting breast ultrasound with increased experience, with a peak at six to 10 years. We also saw better performance when the investigators spent at least 25% of their time in breast imaging -- and most importantly, when they participated in scanning patients and performed at least 20-30 breast ultrasound exams per week."

By Shalmali Pal
AuntMinnie.com staff writer
December 17, 2004

Visit the Women's Imaging Digital Community on December 22 for a second study on ACRIN 6666 investigator training in ultrasound screening of the postsurgical breast.

ACRIN updates multiple breast cancer trials, April 15, 2004