Prediction of frequent emergency care use by those with chronic conditions with machine learning models is not superior to existing approaches
Predicting frequent emergency care use by patients with chronic health conditions with machine learning models does not offer any additional benefit to existing modelling approaches according to the findings of a retrospective analysis by Canadian researchers.
While, not universally accepted, those with at least three or more visits per year have been used to describe a frequent emergency care user. Such individuals often have complex health needs which are not met through primary care provision and consequently their condition deteriorates, leading to an emergency department (ED) visit. Although frequent emergency care users (FECU) represent only a small proportion of the overall population seen in an ED, they do account for a disproportionately large number of visits. Currently, logistic regression models have been used for analysing frequent users in emergency departments. However, the development of machine learning models that can incorporate large amounts of both clinical and non-clinical data, have the potential to help identify FECU individuals. Such models have already been used, for example, in predicting the need for hospitalisation at the time of triage for children with an asthma exacerbation. Nevertheless, no studies have explored the use of machine learning models – and how these compare with logistic regression – to predict frequent emergency care use in adults with chronic conditions.
In the present study the Canadian team retrospectively examined the performance of four machine learning models in comparison to a logistic regression model, for the prediction of frequent ED use in adults with a range of chronic conditions. They identified two cohorts: those who had at least 3 and 5 visits per year. The models used a number of predictor variables including age, gender and residential area and focused on chronic diseases such as coronary artery disease, mental disorders, epilepsy and chronic non-cancer pain. The models were used to predict frequent ED use as a binary outcome, i.e., frequent user or not and the outcomes were compared in terms of the area under the curve (AUC), sensitivity and specificity.
Frequent emergency care user model predictions
The analysis identified 451,775 ED users and of whom, 9.5% had at least three visits per year and 3% five visits.
The AUC for the logistic regression model for frequent users with 3 visits/year was 0.748, giving a sensitivity of 60% and a specificity of 78%. Two of the machine learning models gave a similar AUC (0.749 and 0.744) whereas the random forest model was much worse (AUC = 0.538). For prediction of frequent users with 5 visits/year, the model performance was broadly similar, i.e., machine learning-based models were no better.
Overall, the authors commented on how none of the machine learning models outperformed the logistic regression model and the most important predictor variable was the number of visits in the previous year. The authors did feel that access to more variables could have helped in refining the predictive accuracy of the machine learning models. Nevertheless, they emphasised the need for future work to consider complex non-linear interactions, since in such cases, machine learning models were likely to be superior to existing ones.
Citation
Chiu YM et al. Machine learning to improve frequent emergency department use prediction: a retrospective cohort study. Sci Rep 2023
