The field of ultrasound has witnessed a revolution in the last two decades as hardware miniaturization has wrought seismic changes in how, when, and where sonography can be used. But ultrasound could be on the cusp of a second revolution, with new software applications promising to make sonography even more portable, user-friendly, and intelligent.
Artificial intelligence (AI) is powering many of these new advances. AI is being used most visibly for image interpretation, marking suspicious areas on images or alerting clinicians to patients who need immediate attention. But it's also playing an important role behind the scenes, powering new methods of data reconstruction.
AI is also playing a role in teleultrasound, ensuring access to ultrasound in geographic areas where healthcare workers with ultrasound training are in short supply, integrating with PACS and transferring images safely to the cloud, and acting as triage tools to determine if more investigation is necessary in a given patient.
Like a GPS guide
Augmenting teleultrasound with AI is like giving novice users a GPS to guide them in image acquisition, according to Dr. Partho Sengupta, chief of the division of cardiology and director of cardiac imaging at West Virginia Heart & Vascular Institute.
In locations where there is no experienced provider to do the scanning, telemedicine can be combined with ultrasound to perform echocardiography at a distance. Sengupta said. AI can guide untrained users in how to move the probe on the chest of a patient, thanks to navigational tools embedded in the scanner, so the user arrives at the ideal location to image the heart. Images can then be transferred to a central location for interpretation by an experienced cardiologist.
Dr. Partho Sengupta."I can then answer the questions about the case remotely," Sengupta said. "The key thing is to get the information to the expert and augment their ability to make a decision remotely."
The cardiologist working remotely can then make decisions that can be implemented at the remote site, such as whether an urgent consultation is needed, Sengupta explained.
"It allows for better informed triaging of patients," he said. "We can determine which cases need immediate consultation with a cardiologist and prioritize consultation."
Teleultrasound can even help reduce unnecessary testing, Sengupta noted.
"Many patients are sent for more advanced tests, and there are a lot of costs incurred," he said. "It is a good way to avoid inappropriate testing."
Ultrasound during the pandemic
With the ongoing COVID-19 pandemic, AI-based software can reduce the need for healthcare personnel in the intensive care unit (ICU) to conduct an ultrasound, Sengupta pointed out. In fact, in a recent paper in JACC: Case Reports, a group of researchers from Northwestern University described how they used a commercial software application (Caption Guidance software, Caption Health) to guide novice users in acquiring cardiac ultrasound exams in a COVID-19 ICU.
The novice users, with no formal ultrasound training, were able to obtain point-of-care ultrasound (POCUS) cardiac images using Caption Guidance, which provides real-time guidance to steer the user's transducer position and hand movements to acquire the desired cardiac ultrasound images. The software can also detect ejection fraction with similar accuracy to that of the standard volume-dependent approach, according to the researchers.
"The fewer number of people who are taking care of a COVID-19 patient, the better," said Sengupta, who penned an editorial on the study in the same issue of the journal. "We know that the patients admitted to the ICU and on ventilators [who have COVID-19] have a high chance of cardiac involvement. This is a good tool for understanding that involvement."
AI in perinatal care
Other software applications are going even further, supporting voice and video communication between users in the field and experts located remotely. One such application (Collaboration Live, Philips Healthcare), runs onboard an ultrasound scanner and enables real-time collaboration between users of the machine and experts in a remote location.
For Dr. Michael Ruma of Perinatal Associates of New Mexico, the software raises the bar for the patient experience.
Dr. Michael Ruma."When I make an audio-video link to the machine, I now can communicate with the patient," Ruma said. "I can communicate with the sonographer in real-time with both audio and video."
Ruma sees specific benefits of teleultrasound and Collaboration Live software for perinatal care, permitting him to operate the scanner, drawing arrows on the remote screen and directing patients to what he is looking at. He can freeze images and also look at calculations, as well as share his screen and other information over the remote ultrasound scanner.
"All of those capabilities truly enhance the experience for me and for the sonographer and for the patient. It's exactly as if I walked into a room and talked to the patient," Ruma said. "Physically, I typically go through pictures on the ultrasound machine. I typically hit the trending graphs, show them things on the ultrasound system. I'm able to do that and operate that remotely from wherever I'm at in the world."
One of the software's strengths is its communication and telemedicine capabilities, but if users choose, it also can generate reports as the platform can control the keyboard and the printing function.
Reimbursement with teleultrasound has not been a disappointment for clinicians: Ruma noted that reimbursement for teleultrasound was very close to that of an in-person visit, at between 97% and 98% of in-person fees.
Image integration and QA
Teleultrasound can also confer benefits in terms of credentialing and quality assurance (QA). Teleultrasound images acquired in remote locations -- even by personnel who don't have credentials -- can be moved into a PACS and reviewed by physicians who do.
Dr. Peter Weimersheimer."If a provider who is not credentialed is performing the study, and I am reviewing it using teleguidance in real-time, and I am credentialed, then it becomes something that can be documented in the medical record," said Dr. Peter Weimersheimer, a professor of surgery and director of emergency medicine ultrasound at the University of Vermont Medical Center.
Weimersheimer uses the Butterfly IQ point-of-care ultrasound scanner, and he noted that teleultrasound improves quality assurance by enabling experienced personnel to guide users in the field.
Dr. Peter Weimersheimer supervises an ultrasound-guided nerve block remotely. Image courtesy of Dr. Peter Weimersheimer."What is really cool with teleguidance is that I can actually say, 'Here is what you are doing wrong with your hand position,' " Weimersheimer explained. "It is as if I was at the bedside, but I am five or six hours away."
At the onset of the COVID-19 pandemic, the health network purchased POCUS scanners for every department.
"It has been a game changer in terms of healthcare," he said. "If I can help someone else be better at it (at performing ultrasound), so much the better."
Support diagnostic decisions
But AI software is going beyond just supporting communication and collaboration, and into supporting actual image interpretation. With respect to image analysis, breast imaging and echocardiography are two of the most prominent early applications.
For example, a study presented at the 2020 edition of the Society of Imaging Information in Medicine (SIIM) meeting found that a commercially available software tool (Koios DS, Koios Medical) reduced interpretation time by 49% by classifying suspicious lesions on breast ultrasound.
Also in breast imaging, Samsung Medison's S-Detect software was also found recently to help decrease false-positive rates for masses detected on screening breast ultrasound exams. In a study published in January, researchers from South Korea shared how a nomogram that incorporates AI-based quantitative analysis offers the potential to reduce unnecessary biopsies. More than 40% of false positives could potentially be eliminated without impacting sensitivity, the authors concluded.
A number of other AI developers are targeting the market for cardiac image analysis. One firm has developed AI-based software that chooses the best cardiac ultrasound views from standard echocardiogram exams (LVivo Seamless, DiA Imaging Analysis). The company also offers applications for automated analysis of the heart's right ventricle (LVivo RV), as well as for automated bladder-volume measurements (LVivo Bladder).
DiA's new AI-based ultrasound analysis tool LVivo RV. Image courtesy of DiA.Some software developers are specifically targeting the fast-growing POCUS market, where some believe the need for AI by novice users may be even greater than that of expert users. One firm (EchoNous) has leveraged machine learning to provide real-time, automatic image labeling, grading, and guidance, as well as image guidance that shows physicians how to fan and rotate probes.
Cardiac ultrasound is also the focus of RSIP Vision, which has developed a range of tools for cardiac ultrasound applications, such as coronary artery segmentation and the evaluation of heart function.
Another firm, Ultromics of the U.K., is developing software that will help customers get more use out of existing equipment rather than having to purchase new scanners. The company provides applications for cardiac ultrasound, such as EchoGo Core for strain echo analysis and EchoGo Pro for diagnosing coronary artery disease.
Victoria Pennetti.Instead of buying new hardware, practices can utilize AI software to stay current and minimize operational costs, according to Ultromics' customer Victoria Pennetti, a sonographer and CEO/founder of Ultrasound Solutions.
"This gives us the ability to use new technology without having to upgrade our equipment," Pennetti said.
Clinicians reading images have been able to read and interpret from whatever location they are based, noted Pennetti.
Other ultrasound AI algorithms cleared by the U.S. Food and Drug Administration include the following:
- EchoMD AutoEF, Bay Labs: Automated quantification of ejection fraction
- Automated Real-Time Intelligent Assistant (ARIA), Medo.ai: Detection of hip dysplasia on ultrasound exams in infants
- Augmented Vascular Analysis (AVA), See-Mode Technologies: Assistance in interpreting and reporting vascular ultrasound studies
- AmCAD-UT Detection, AmCAD BioMed: Analysis of thyroid ultrasound images
- QVCAD, QView Medical: Image processing and computer-aided detection on 3D automated breast ultrasound exams
- Auto 3D Bladder Volume Tool, Butterfly Network: Bladder volume measurements
Some AI developers are even focusing their efforts on other software developers rather than end users. One company, WaveBase of Canada, is creating tools that will allow users to download algorithms from a PACS or cloud and deploy them bedside. One such tool automatically calibrates from pixels to centimeters, a function that is critical for many AI algorithms.
WaveBase technology was used in the development of AlveolAI's lung ultrasound B-line classification tool, shown here correctly identifying hydrostatic pulmonary edema (code: Smooth) at the bedside. Image courtesy of WaveBase.The company is also developing a tool for masking ultrasound images, such that data that surrounds the image on the screen is hidden from view, noted Peter Holmes, CEO of WaveBase.
"It would mean only the ultrasound images are transmitted and not personal health information," he said.
Fine-tuning AI solutions -- and what's to come
As the software revolution transforms ultrasound, automation is driving innovation in AI. What happens next is anyone's guess: Sengupta even sees a marriage of software and hardware in terms of a robotic arm with a transducer that could be remotely controlled to perform an ultrasound scan.
"There are a lot of creative things being done right now in AI," Sengupta said.
