Accurately predicting how long patients may have to wait for their imaging exams is key for both practice management and patient satisfaction. Machine learning may be able to help, according to a study published online October 24 in the Journal of the American College of Radiology.
A team led by Oleg Pianykh, PhD, of Massachusetts General Hospital (MGH) tested a variety of machine-learning models to determine which was most effective in predicting patient wait and facility delays for CT, MRI, ultrasound, and x-ray exams. They found that, as is often the case, simpler is better.
Oleg Pianykh, PhD, from MGH."There's research on how machine learning improves image processing and radiologist workflow, but not so much on how it might improve the operational side of radiology," Pianykh told AuntMinnie.com. "We found that one of the simplest algorithms, elastic net, proved the most effective in helping to predict patient wait times and exam delays."
Positive predictions
Having to wait for exams or experiencing appointment delays can quickly lead to unhappy patients -- which can have long-term negative effects on a practice, Pianykh said.
"The accuracy of wait time or appointment delay predictions has psychological effects on patients," he said. "If you tell a patient that the wait will be 20 minutes and instead it's only five, they'll be happy, but if you tell them it will be five and it turns out to be 20, they're going to be upset and they may not return. We had already been using predictive models that were somewhat effective, but we wanted to develop one that could accurately handle more complexity."
The study included CT, MRI, ultrasound, and radiography exams performed between July 2016 and January 2017 at MGH; of all the modalities, only radiography had walk-in exams. The researchers used the hospital's RIS to extract nine exam parameters:
- Patient arrival time
- Exam start and end times
- Time of first image acquisition
- Exam code
- Exam description
- Scanner name
- Modality
- Hospital division that conducted the exam
- Scheduled appointment time (for CT, MRI, and ultrasound exams)
The researchers defined "delay time" for scheduled appointments as the time between the scheduled exam time and the time the first image was acquired; "wait time" was the time between patient arrival and first-image time for walk-in exams.
Using the original RIS data, Pianykh and colleagues also developed a list of 40 "predictors," or variables, that could contribute to delays or wait times. General predictor categories included examination-specific details, proximity to a holiday, and the arrival order of patients.
The researchers then trained 10 different types of machine-learning algorithms to test the data and find the best algorithm for all four modalities. Algorithms included neural network, random forest, support vector machine, elastic net, multivariate adaptive regression splines, k-nearest neighbor, gradient boosting machine, bagging, and classification and regression tree. The group evaluated each model by computing its root-mean-square error (or RMSE, which is a way to measure accuracy) and R2 value (a statistic that describes how well a model fits the task).
Pianykh and colleagues found that the elastic net and gradient boosting machine algorithms had the best RMSE and R2 values and, therefore, were most effective at accurately predicting wait times and appointment delays. In the end, the team chose the elastic net algorithm because it was more computationally efficient, he said.
"An elastic net algorithm is a more advanced form of classical linear regression and is capable of learning the most important predictors from high-dimensional, noisy data," he told AuntMinnie.com.
The most important predictors for wait times and delays were patient queue length, exam queue length, order in which the most recent patient arrived in relation to other patients with the same appointment time, and median exam time of the five most recent exams (especially MRI and ultrasound).
Machine learning's computational power makes it an excellent choice for recognizing and predicting "sophisticated and noisy phenomena," such as waiting times, the researchers concluded.
"We found that it's not necessarily the most complicated machine-learning models that produce the best results," Pianykh said. "But it's definitely true that machine learning can improve practice management and, therefore, patients' healthcare experience."
