HONOLULU - Researchers from California produced photorealistic 3D models of the breast by applying cinematic rendering to 3D transmission ultrasound images. The potential clinical applications of these models were the subject of a Wednesday presentation at the American Roentgen Ray Society (ARRS) annual meeting.
Highly touted for its photorealism, cinematic rendering is picking up steam as a method for enhancing the visualization of conventional medical images, especially CT scans. Could the advanced technique also function with ultrasound images?
The integration of several different disciplines, from artificial intelligence (AI) to image reconstruction, has paved the way for successful cinematic rendering of breast ultrasound, presenter Dr. Rajni Natesan told session attendees. Natesan is a visiting scientist at the University of Texas MD Anderson Cancer Center in Houston and chief medical officer of QT Ultrasound.
"We think there are benefits to these types of renderings for both physicians and patients in the setting of communication and diagnoses and surgical planning, and we're looking in future studies at its utility for diagnostic purposes, compared with what's available today," she said.
Photorealistic breast models
For decades, visual effects artists have relied on cinematic rendering to enhance the image quality of movies. It is only in the past few years that clinicians have made a concerted effort to apply this technique to medical imaging. Preliminary research has demonstrated the potential of cinematic rendering to provide clinicians with insight into complex cases involving, for example, aortic complications, soft-tissue tumors, and traumatic brain injury.
Nearly all of this work has revolved around applying the concept of global illumination -- the propagation of thousands of rays of light -- to a dataset of medical images such as CT scans. In the current study, Natesan and colleagues demonstrated how they applied the same idea to images of the whole breast acquired with 3D transmission ultrasound.
The researchers began by obtaining the volumetric data of in vivo 3D transmission ultrasound images of five different breasts. Then they used a proprietary machine-learning algorithm to automatically segment the images and color code them by tissue type, including skin, ducts, glandular tissue, connective tissue, and fat. The algorithm also revealed the speed of sound and reflection values for the distinct tissue types.
Finally, they input these data into a variety of separate software for cinematic rendering:
- Open-source compositor software (Natron, Inria) for 3D rendering
- Computer software (Maya, Autodesk) for manipulating volume density, light scattering, and transparency of the 3D volume
- Computer software (Arnold, Autodesk) for modeling light reflections and interactions on the different types of tissue
In effect, this protocol enabled the researchers to generate highly detailed maximum intensity projections (MIPs) of the whole breast that could be viewed from any angle. They were also able to separate individual components of breast anatomy and pathology for 3D visualization as maximum intensity projections.
Unlike more common cinematic rendering techniques, this extensive process for "volumetric cinematic rendering of ultrasound tomography was powered by multiple disciplines, including tomographic image reconstruction, machine learning for tissue-dependent segmentation, and simulation of light-tissue interaction," Natesan said.
Tool for communication
The group is currently conducting several pilot studies investigating the utility of cinematically rendered 3D transmission ultrasound images for diagnostic purposes, compared with conventional ultrasound, mammography, and volume renderings.
Preliminary research has shown a high correlation between the cinematic renderings and standard imaging modalities for breast lesion localization, and the technique may serve as a promising alternative for identifying malignant lesions in the future, Natesan noted.
In one case, a 56-year-old woman who presented with a palpable mass in the upper central portion of the right breast had nodularity confirmed with mammography and ultrasonography. Cinematically rendered 3D transmission ultrasound images allowed the researchers to manipulate the data and segment out nodules and distinct tissue types for isolated examination, further clarifying their understanding of the pathology.
Another benefit of the group's cinematic rendering technique is that the segmented images are compatible with other forms of 3D visualization. Clinicians can even convert the segmented images into stereolithography (STL) file format for use in 3D printing.
The highly photorealistic breast model can serve as an important tool for enabling effective communication between providers and patients and between radiologists and referring physicians, Natesan said. In the future, it may also offer advantages in presurgical planning and intraoperative guidance for breast-related procedures.
