How augmented reality can transform medical imaging

From reading rooms filled with lightboxes and x-ray film to 3D holographic images of patient anatomy, we've seen incredible technological advances in radiology just over the past 15 years. I believe we are on the cusp of yet another exponential transformation in the way we as radiologists practice our specialty.

Dr. Jesse Courtier from the University of California, San Francisco. Within the past year, we have seen a tremendous increase in excitement about the potential for both augmented and virtual reality (AR/VR) applications in the field of diagnostic imaging. This is due in part to rapid advances in computer processing capabilities that now allow the type of real-time rendering that would not have been possible a decade ago. The possibilities of moving from 2D representations to true 3D patient-specific holographic models are extremely exciting to many of us.

Augmented reality applications, which I have been working with over the past year, have been particularly exciting for my clinical colleagues. "Wow!" has been the most common reaction from first-time users of my HoloLens app, Radiology with Holographic Augmentation (RadHA). To see the anatomy represented in a 3D way is more realistic and closer to what they actually encounter when in the operating room. The possibilities of superimposing patient-specific 3D medical objects onto a real-world background have generated new ideas for patient and trainee education, presurgical planning, and even intraoperative use.

Our societies have also taken notice. There will be a new AR/VR showcase space alongside the 3D printing space at RSNA 2017, and there are plans for an AR/VR session at a future meeting of the American Roentgen Ray Society (ARRS). This will likely lead to a collaboration between those in academia and industry to allow the continued growth and expansion of these applications in radiology. It is then possible that in the future our primary means of interpreting images will be in 3D, with scrolling through 2D stacks of images becoming a thing of the past.

This will not be the first time radiology has undergone rapid technological change (which, honestly, is what makes the field so exciting to me). The widespread incorporation of PACS into radiology workflow in the early 2000s allowed images to be rapidly digitized and interpreted. This meant that radiologists could read more studies and read them faster.

The RadHA app uses Microsoft's HoloLens virtual reality technology to display 3D CT images, overlaid on a real-world background.

Radiologists had to adapt to faster workflows and retrain their minds from reading images on printed film to scrolling through stacks of hundreds of images in a few minutes. This was accomplished, and as new trainees started after the adoption of PACS, the transition appeared almost seamless.

Similarly, I believe we as radiologists should be heavily involved in the development of emerging technologies such as augmented and virtual reality at the earliest stages.

At a time when our future as a specialty is being questioned due to the advent of technological advances such as artificial intelligence, we now, more than ever, must work diligently to stay at the forefront of technical innovation. With the development of standardized training protocols for 3D image creation and interpretation and AR/VR software optimization, we are the best positioned to lead these fields forward and to continue to make radiology's future a bright one!

Dr. Jesse Courtier is an associate professor of pediatric radiology at the UCSF Benioff Children’s Hospital with a special interest in augmented reality. He has co-developed the augmented radiology application RadHA for the Microsoft HoloLens. This article is adapted from a posting on LinkedIn, with permission.