An immersive virtual reality (VR)-based test can distinguish between operators with varying levels of thoracic ultrasound skill, a Danish study published January 6 in Ultrasound in Medicine & Biology found.
Researchers led by Anders Nielsen from Odense University Hospital in Svendborg found that between novice, intermediate, and expert ultrasound users, the experts had the highest test scores while the novice users had the lowest scores.
“This [test] enables a standardized, objective, and evidence-based approach to assessment of thoracic ultrasound skills,” Nielsen and co-authors wrote.
With ultrasound being a user-dependent imaging modality, operator skills are important for diagnostic accuracy. Previous research suggests that immersive VR could play a role in advancing hands-on medical education, radiology included. However, the researchers noted a lack of data on its educational use in thoracic ultrasound.
The Nielsen team developed its immersive VR test to assess thoracic ultrasound skills among clinicians of varying skill levels. It also gathered evidence in validating its test and created a pass/fail score.
To develop the test, an expert panel used information and recommendations from the European Respiratory Society’s (ERS) protocol on thoracic ultrasound. This included a tutorial and two clinical cases regarding pleural effusion and interstitial syndrome.
The team used an immersive VR platform from VitaSim, based in Odense, Denmark. Through the platform, the team made available four anterior zones, four lateral zones, and six posterior zones available for examining and diagnosis decision-making. Clinicians received one point for each correct response to test questions.
The researchers collected data during the ERS’ 2022 congress in Barcelona, Spain. They included test data from 13 novice ultrasound users, 22 intermediate users (1-50 thoracic ultrasound exams), and 11 experienced users (> 50 exams).
The team reported that the novice users scored the lowest while the experienced users had the highest scores. Additionally, it found a Cronbach's alpha value of 0.86 for the entire test, indicating high consistency.
| Average scores of thoracic ultrasound users on immersive VR tests (range, 0 to 10) | ||
|---|---|---|
| Type of reader | Case 1 | Case 2 |
| Experienced | 8.7 | 8.5 |
| Intermediate | 7.3 | 6.7 |
| Novice | 5 | 4.5 |
The team found significant differences between the novice and intermediate users for Case 1 (p = 0.007) and Case 2 (p = 0.02). It also reported significant differences between the intermediate and the experienced groups for Case 1 (p = 0.04) and Case 2 (p = 0.019) and for Case 1 (p < 0.001) and between the novice and experienced groups for Case 2 (p < 0.001).
Item analysis showed little difficulty in choosing the right probe and none of the test items was reported to be too difficult. The team also established a pass/fail score of 7 points in each case.
The study authors suggested that future immersive VR studies could consist of a wide variety of educational tools that are structured and evidence-based. Depending on the educational setting, the institution or participant could then pick out the most convenient and efficient educational tool, they added.
“This standardized portable test could be a prerequisite in a structured simulation-based mastery learning program,” the authors wrote.
The full study can be found here.
