Researchers at the University of California, San Francisco (UCSF) created holograms that have comparable dimensions to corresponding CT scans using proprietary augmented reality (AR) software. They plan to present their work at the upcoming American Roentgen Ray Society (ARRS) meeting in Washington, DC.
In the pilot study, the group led by Dr. Jesse Courtier performed a series of measurements on five augmented reality holographic models based on CT phantoms. The resulting measurements of their holograms were as accurate and precise as manual and PACS measurements of the same CT phantoms.
"Our findings lay the groundwork for developing accurate holograms, which will help establish a foundation for their clinical use," Courtier told AuntMinnie.com. "By overcoming hurdles like this, we can begin considering the future use [of augmented reality] live in the operating room, where distances within a few tenths of a millimeter are important."
Lifelike holograms
Courtier and colleagues have been exploring the clinical application of augmented reality through their proprietary Radiology with Holographic Augmentation (RadHA) software, which works in conjunction with the Microsoft HoloLens wireless headset to display CT scans and MR images in 3D on any real-world background.
"Our prototype augmented reality software allowed for the visualization of medical images in a more natural way," he said. "So we wanted to know how accurate these projections could be, and if they could be even more lifelike."
To determine the fidelity of their models to real-life measurements, they collected CT scans of five distinct quality control phantoms and converted these scans into 3D models through the RadHA software. They were then able to visualize holographic replicates of these models by wearing the HoloLens headset. With the headset on, they manually measured several preselected sections of each of the models in three planes (x, y, and z) using calipers.
When they compared this information with manually collected standard measurements, they found there was no statistically significant difference between these gold standard numbers and those they took of the projected AR hologram models. The holography measurements were also nearly identical to the automated PACS measurements of corresponding CT scans.
Measurements of augmented reality holograms match standard measurements. Image courtesy of Dr. Jesse Courtier.These initial findings demonstrate that current AR technology is capable of producing reliable holograms from CT scans and could be used for educational, training, or research purposes, according to the investigators.
The results further highlight the potential of augmented reality as an "alternative to virtual reality -- where users are digitally blindfolded -- as well as 3D printing, which typically has a long turnaround time and a high cost," Courtier said. "I can make these [holographic] models within 25 minutes and include new iterations in less than two hours, and even change the colors of the models on the fly."
"I was impressed by how accurate measurements of the holograms were," said presenter Jesus Uribe, a third-year medical student at UCSF. "To be able to participate in that [research] was really inspiring for me as a medical student and as someone interested in a career in radiology."
'Starting point'
What are some of the possible clinical applications of augmented reality holograms?
The researchers have already begun incorporating their AR technology into presurgical planning, especially to support communication during presurgical planning conferences.
"The augmented reality device is another tool for radiologists to use to relay information that we have in our minds -- to take 3D reconstructions of 2D images and display them in a way that's more natural to what the surgeon sees in the operating room," Courtier said. "We've had a lot of positive feedback from surgeons. Being able to see things in a new way is exciting for them."
Though only one person at a time can wear the headset, the holograms visible to the user can also be displayed simultaneously on a separate screen, allowing everyone to virtually walk through the case together, he said. This enables a seamless integration into the current workflow: The surgeons can look at the medical images alone first and then examine the holograms with radiologists and the rest of the team in a multidisciplinary conference.
With the holographic models now proving to have highly accurate measurements, physicians might even be able to rely on the measurements of tumors or vessels collected before surgery to help guide the operation, he noted.
"This study is a starting point showing that we have the ability to use [augmented reality holographic models] intraoperatively," he said. "Our next plan is to obtain measurements of holograms based on MR images. We've already received a grant for this research."
