As advanced visualization techniques such as augmented reality (AR) continue to proliferate, radiologists need to become familiar with these technologies, according to an article published online October 28 in 3D Printing in Medicine.
In a technical note, researchers from New York University (NYU) Grossman School of Medicine discussed the benefits of AR and shared their workflow for creating patient-specific AR models. They also detailed how these AR technologies are being used to help plan urologic surgical procedures.
"Visualizing complex medical imaging data as real 3D anatomical models in AR can facilitate understanding of complex surgical anatomy by helping to depict the relationship of dominant tumors to key anatomic structures, allow surgeons to prepare for complex surgical cases, and enable patients to better understand their disease and treatment plan," wrote the authors led by first author Nicole Wake, PhD, also of Montefiore Medical Center.
Each advanced visualization method -- AR, virtual reality (VR), and 3D printing -- has its advantages. 3D printing provides tactile feedback, while VR offers a fully immersive experience, according to the researchers. And AR facilitates real-time surgical navigation.
"In the future, it is expected that surgeons will be able to request 3D models to enhance conventional medical image visualization and that the model type will depend on the procedure type, time frame, surgeon preference, and available utilities," the authors wrote.
AR -- in contrast to VR or 3D-printed models -- can be co-registered to the surgical anatomy during the surgical procedure and provide the surgical team with a real-time road map. In the future, AR technologies could be integrated into operating room systems to facilitate patient-specific surgical rehearsals and provide real-time surgical navigation, according to the researchers.
At their institution, anatomic 3D models are created for use in AR on the HoloLens (Microsoft) device. As AR devices don't currently accept DICOM images as input, a few preparatory steps are required.
Following acquisition, images are segmented using the Mimics software (Materialise) to delineate the appropriate anatomical regions of interest (ROIs) to be included in the AR visualization. Next, computer-aided design modeling is performed. Each ROI is converted to a 3D surface "mesh," which is then exported in the STL file format -- the most common format used for 3D printing, according to the researchers.
The 3-matic software (Materialise) is then used to perform "marginal local smoothing" to minimize the pixelated appearance of the images. In addition, contours are created and overlaid onto the DICOM images. This ensures that the models accurately reflect the anatomy and that all anatomical structures can be viewed simultaneously in 3D, according to the researchers.
"It is important to note that different degrees of wrapping and smoothing meshes in [postprocessing] programs may cause a loss of detail," the authors wrote. "In addition, it is essential to check for accuracy of both segmentation and post-processing as a last step before printing or utilizing these models in AR or VR."
In the last preparatory phase, the Unity gaming engine is used to import the 3D anatomical models into the HoloLens. Full-color holograms can then be rendered in AR on the device.
Their institution is currently performing a prospective study to evaluate the impact of 3D-printed and AR models on patient care.
"These models are utilized by surgeons and shown to patients preoperatively," the authors wrote. "If desired, surgeons could also use the AR models intraoperatively."
