Microsoft's Kinect motion-sensing technology isn't just for controlling video games. It can also enable surgeons to view and manipulate medical images within the patient's sterile field in the operating room, according to research from Sunnybrook Health Sciences Centre in Toronto.
Thanks to internally developed software that allows the Kinect to access the institution's PACS, surgeons can hail the Kinect by using a defined hand gesture at any point in the operation. Then, using other hand gestures, they can perform tasks such as moving images and modifying brightness and contrast, said Dr. Matt Strickland, a general surgery resident, engineer, and one of the co-developers of the system.
"You can do everything that you can do with the mouse," he told AuntMinnie.com. "This is a solution to the very unique barrier that exists in the operating room and the interventional radiologists' room."
Access to medical imaging can be problematic within the sterile field of an operating room. But a team of engineers, residents from the University of Toronto, and Sunnybrook surgeons has developed a hardware and software combination that allows Kinect to be used to control a PACS workstation.
After Kinect was hacked by computer programmers following its release in November 2010, the open-source community released a basic library to access the streams of data coming out of the Kinect, Strickland said. That opened the door for its possible use to access medical images on the PACS.
"Two of my engineering friends and I decided that this would be a perfect tool to solve this problem I had been looking at for quite some time," he said.
Kinect employs an RGB camera that searches around a room, creating 3D images of what it sees. The developers' software then recognizes human bodies and their signs.
Once someone in the room hails the Kinect, he or she gets assigned "master status" and gains control of the rest of the device, he said. An added piece of hardware allows for seamless installation via a USB connection.
"You just plug it in like you would any mouse," he said. "So this way you don't have to install any software on the PACS computer. You literally can walk up and set the Kinect down on the counter."
Dr. Calvin Law of Sunnybrook Health Sciences Centre demonstrates how surgeons can use Kinect to manipulate patient images. Image courtesy of Doug Nicholson/MediaSource.Pilot research involved developing the best gestures for various tasks, Strickland said.
"In the operating room, gestures that you would normally want to use, such as putting one hand below your waist or one putting one hand behind your back, are off limits," he said. "These are the challenges we're trying to work out."
The system is undergoing usability testing at Sunnybrook to determine how best to incorporate it into surgical practice, Strickland said. It's currently being used by surgeons, but interventional radiologists have also expressed interest in the system, he said.
Surgeons have responded well to the new technology, according to Strickland. One of the big advantages is its ability to facilitate rechecking of images, a task that currently may involve the surgeon having to break sterility.
"Surgeons are going to be enabled to double- and triple-check [images], reassure themselves, and really take another look at the road map that's offered via the PACS as they guide their surgery strategy," he said.
The system can be seen in action here.
Once the pilot phase is complete, the system will be deployed first to other hospitals in Toronto, with other institutions in the U.S. and Canada to follow in the future, Strickland said. Talks are under way to determine a commercialization strategy.
A research study is also being put together to quantify the system's use and its time savings, he said.
