For all its proven value, 3D printing is just one of many ways to look at 3D data -- and it's a rather rigid one at that. For clinicians, understanding a patient's medical condition may require more than a printed skull or aneurysm. In these cases, a holographic dataset can be helpful.
Advanced visualization firms are pursuing holographic data visualization as a way to provide physicians with an immersive experience with medical imaging datasets that could be either a complement or an alternative to 3D printing. One such firm is TeraRecon, whose CEO, Jeff Sorenson, spoke on the topic at the 3DHeals conference in April in San Francisco.
Sorenson sees a future in which physicians may collaborate in real-time holographic environments, and then choose to print out in 3D only those elements with which they need to physically interact.
High-resolution for all
While the stereolithography (STL) file format forms the basis of 3D printing, its resolution is so low as to make it worthless for most other applications. Therefore, TeraRecon has replaced STL with a Microsoft-compatible file format that is compatible with full-resolution data files for both printing and virtual visualization of imaging data. The company does all the printing itself via its 3D Print Packs, a kind of printing voucher sold to clients.
For virtual 3D visualization, those same full-resolution files are loaded onto Microsoft's virtual reality holoport platform with a $5,000 HoloLens virtual reality headset. HoloLens can use 3D data from multiple sources, and it's a great tool until you load it up with a traditional low-resolution file such as an STL file used to print a 3D heart or kidney. The results can look very rough, Sorenson said.
"It's not good for the same reason that your 3D print has low resolution," he said. "The value of that file was destroyed when you moved it into STL and limited yourself to a number of polygons. And then you say, 'Look, when I zoom it up, it's terrible.' "
In full-resolution 3D, every pixel displayed on the platform is part of the target anatomy -- say the inner ear from a head CT volume -- rather than part of a low-resolution dataset zoomed beyond recognition. 3D viewing doesn't have to be a low-resolution exercise, he said.
"We limit our thinking because of the technology that's in front of us," Sorenson said.
Printing as an option
Sometimes you need a 3D print and sometimes you don't, according to Sorenson. 3D prints offer a tactile model that can be very helpful for surgeons and trainees. But they cost time and money that you don't spend if you don't print.
Without 3D printing, you can still move the 3D image volume around in front of you, collaborate with it, and enjoy instant access in augmented reality.
With 3D printing, people talk about the potential for collaboration, he said, but how well does that work?
"Are all of the experts really in the same room at the same time and ready to look at the same model at the same time to prepare for surgery?" Sorenson asked.
What makes more sense is to have a holographic experience first and then print those pieces that need printing, he said.
"It's no different than what you do with paper," he said. "There are some you want to print and sign and hand to someone else."
Image analysis
Full-resolution files can also facilitate real-time interactions with machine learning, Sorenson said. Inside the holoport application, the clinician can access advanced image processing apps to gather additional information using image analysis driven by artificial intelligence (AI). The results can be invoked within HoloLens, which is equipped with sound, or within a 2D space.
For now, TeraRecon's AI-based image functions are limited to two: automatic labeling of anatomy (Atlas Maker) and automatic image organization (Image Organizer) to present images "the way the practitioner likes to see them," Sorenson told AuntMinnie.com. The apps learn from human interaction, so that each subsequent iteration is a little closer to what the practitioner would have created for him or herself.
The Atlas Maker AI application performs automatic labeling of CT angiography images of the abdominal aorta. All images courtesy of TeraRecon."Name the anatomy, and it will orient the viewer to that anatomy and be in volumetric space orientation with image processing done automatically," he said. "As for image organization, every clinician likes to see the images in his own way, so if the clinician doesn't like the image layout and decides to change it, the algorithm learns from the change."
In collaboration with TeraRecon's 3D printing and holography partner Vizua, holographic images are available anywhere the viewer might want them -- hovering in virtual space and ready for manipulation without the need for a monitor.
Together, the embedded AI functions combine and process complex multistep procedures to the point of "knowing what to do before you ask," Sorenson said.
Image Organizer demonstrates image classification in a cardiac MR exam showing match confidence.Teaching anatomy
TeraRecon is collaborating with the Cleveland Clinic to provide physician training and anatomy training with HoloLens in an immersive experience, Sorenson said. TeraRecon's Within Image Analysis (WIA) Cloud AI engine will perform tasks such as looking at an axial image of a lumbar spine for areas of low density that signal a high risk of osteoporosis. Another AI engine is a locator that makes a noise every time it encounters a fine fracture.
"You can start to combine these engines into ensembles and get very sensitive, very specific results with machine learning," he said.
The best part about machine learning is that it gets smarter and makes better decisions the more you use it. WIA Cloud is a closed-loop system that can be used with HoloLens and 3D printing when needed. The company is putting substantial resources into AI. On June 1, TeraRecon announced the acquisition of AI firm McCoy Medical Technologies, a move aimed at enhancing the use of third-party computer vision and artificial intelligence applications.
"To us, the future is really about very simple app-specific solutions, and that is a completely complementary concept to 3D printing," Sorenson said.
Computing has finally reached the point where the image can be volumetric and understandable by everyone, "and now we're coming up with the definition of a viewer that's not a radiology interpretation viewer or a DICOM file, but a look at medical images in an understandable way," he said. "We are at a tipping point where images are more important than anything else. The next step is to evolve the viewer so that you don't have to be an expert to view and understand these images."
