SAN FRANCISCO - A new data reconstruction technique called cinematic rendering that creates photorealistic images was discussed in a Monday session at the International Society for Computed Tomography (ISCT) symposium. The technique is like "3D on steroids," according to Dr. Elliot Fishman, who is testing it at Johns Hopkins University.
Fishman displayed a series of lifelike images generated with the technique, wowing both ISCT 2017 attendees as well as the show's course director, Dr. Geoffrey Rubin of Duke University Medical Center. While the clinical value of the images is yet to be determined, Fishman speculated that they could ultimately help 3D achieve its full potential.
"With the original 3D, everything we do is to try to provide people with a better understanding of what they're looking at. We have these great scanners, and the datasets are so good, but things are so complicated and people have a hard time understanding them, both radiologists and nonradiologists," he said. Cinematic rendering is "something we've been working on for 30 years; it's just taking it up one more level."
Awe-inspiring images
For the past 30 years, 3D visualization has been based on data reconstruction using volume rendering algorithms, which produce images that are awe-inspiring but still recognizable as computer-generated. Some of that appearance is due to the fact that 3D software reconstructs data as if there were a single fixed virtual "light source" illuminating the volume of interest externally.
New cinematic rendering techniques instead employ many more virtual light sources, resulting in images that are almost indistinguishable from real anatomy. Cinematic rendering was discussed in an article published in 2012 in Eurographics; Kroes et al explained how the technique included a lighting model that simulated multiple light sources, both inside and outside the volume of interest.
"If you put all of that together, can you change the quality of the reconstruction to make [images] more photorealistic?" Fishman asked ISCT 2017 attendees. "Although we do a great job with classic volume rendering, the question was, could you do better?"
The principles explained by Kroes et al were incorporated into a cinematic rendering technique developed by Siemens Healthineers and made available as part of a software package called syngo.via Frontier. Fishman began experimenting with the software about a year ago at Johns Hopkins, he said.
Fishman estimated that in the year he's been working with the software, he's created about 1,000 cases. Typical clinical use cases include organs for oncology imaging, complex vascular structures such as aortic dissections and aneurysms, and musculoskeletal/orthopedic work.
In his ISCT 2017 presentation, Fishman explained how cinematic rendering results in subtle changes to details that can be visualized in CT images. For example, in a patient with cirrhosis with portal hypertension, there are visible details such as collateral vessels and their relationship to the liver and abdominal wall and soft tissue. He showed how cinematic rendering demonstrated the different texture of the organ between a patient with a normal organ and one with pancreatic cancer.
Cinematic rendering creates lifelike photorealistic images. Image courtesy of Dr. Elliot Fishman.Fishman and colleagues have written a paper on their experiences with cinematic rendering, to be published in the August edition of the American Journal of Roentgenology.
While cinematic rendering does require more computing power, it's demands are not too onerous, Fishman said. He uses an off-the-shelf computer with a special graphics processing unit (GPU) board from hardware developer NVIDIA, costing a total of about $8,900. The software works with all conventional CT scans.
The biggest challenge is the extra effort involved in creating cinematic-rendered images, and Fishman noted that there currently isn't a reimbursement code for the work radiologists might spend on it. The process right now is slow and requires a lot of computer work, but the arrival of faster GPU boards from NVIDIA should improve things, he said.
Another issue is variability in presets used to create shortcuts for image processing. A preset that works perfectly for a cinematic rendering in one patient could be horrendous in the next one. Or, a preset that works well in the axial plane doesn't work in the coronal plane.
Ultimately, Fishman sees cinematic rendering as a logical evolution of 3D in terms of making medical images easier for both radiologists and nonradiologists to interpret. The technique could even play a role in the development of deep learning, he believes. Cinematic rendering employs heavy use of texture maps, and texture maps could be analyzed by deep-learning algorithms to detect and characterize pathology, in particular subtle tumors.
In the question-and-answer portion of the morning's sessions at ISCT, Rubin asked Fishman about the potential of the technique in the virtual reality setting, such as surgeons wearing Oculus headsets using data gloves to manipulate cinematic renderings of images. This could deliver the same experience as a 3D-printed model, without having to print the model, Rubin noted.
"I think the augmented reality ... is going to blow everything else away," Fishman said. "It's great to have something in your hand if you're a medical student, but the stuff I've seen from Oculus ... it's unbelievable, and it's hard to imagine. The people who want to look at things in 3D, that's going to be the way they're going."
