Predicting frequent emergency care use with machine learning no better than existing models

10th February 2023

Prediction of frequent emergency care use by those with chronic conditions with machine learning models is not superior to existing approaches

Predicting frequent emergency care use by patients with chronic health conditions with machine learning models does not offer any additional benefit to existing modelling approaches according to the findings of a retrospective analysis by Canadian researchers.

While, not universally accepted, those with at least three or more visits per year have been used to describe a frequent emergency care user. Such individuals often have complex health needs which are not met through primary care provision and consequently their condition deteriorates, leading to an emergency department (ED) visit. Although frequent emergency care users (FECU) represent only a small proportion of the overall population seen in an ED, they do account for a disproportionately large number of visits. Currently, logistic regression models have been used for analysing frequent users in emergency departments. However, the development of machine learning models that can incorporate large amounts of both clinical and non-clinical data, have the potential to help identify FECU individuals. Such models have already been used, for example, in predicting the need for hospitalisation at the time of triage for children with an asthma exacerbation. Nevertheless, no studies have explored the use of machine learning models – and how these compare with logistic regression – to predict frequent emergency care use in adults with chronic conditions.

In the present study the Canadian team retrospectively examined the performance of four machine learning models in comparison to a logistic regression model, for the prediction of frequent ED use in adults with a range of chronic conditions. They identified two cohorts: those who had at least 3 and 5 visits per year. The models used a number of predictor variables including age, gender and residential area and focused on chronic diseases such as coronary artery disease, mental disorders, epilepsy and chronic non-cancer pain. The models were used to predict frequent ED use as a binary outcome, i.e., frequent user or not and the outcomes were compared in terms of the area under the curve (AUC), sensitivity and specificity.

Frequent emergency care user model predictions

The analysis identified 451,775 ED users and of whom, 9.5% had at least three visits per year and 3% five visits.

The AUC for the logistic regression model for frequent users with 3 visits/year was 0.748, giving a sensitivity of 60% and a specificity of 78%. Two of the machine learning models gave a similar AUC (0.749 and 0.744) whereas the random forest model was much worse (AUC = 0.538). For prediction of frequent users with 5 visits/year, the model performance was broadly similar, i.e., machine learning-based models were no better.

Overall, the authors commented on how none of the machine learning models outperformed the logistic regression model and the most important predictor variable was the number of visits in the previous year. The authors did feel that access to more variables could have helped in refining the predictive accuracy of the machine learning models. Nevertheless, they emphasised the need for future work to consider complex non-linear interactions, since in such cases, machine learning models were likely to be superior to existing ones.

Citation
Chiu YM et al. Machine learning to improve frequent emergency department use prediction: a retrospective cohort study. Sci Rep 2023

Study investigates chronic health conditions and COVID antibody response after second jab

6th August 2021

After a second vaccination, a real-world analysis showed that a quarter of those with chronic health conditions had no detectable COVID antibodies.

Vaccination against COVID-19 is critical to controlling the pandemic. Although the currently available vaccines, are very effective, it is still possible to become infected after receiving both vaccination doses. In the US, for instance, the Centers for Disease Control and Prevention, reported in May 2021, that there had been 10,262 COVID-19 vaccine breakthrough infections. These occurred in patients with a median age of 58 years and while 10% of these patients were hospitalised, fortunately only 2% died. Furthermore, in a recently published study among 1497 healthcare workers, 39 experienced a breakthrough infection and around the time of infection, neutralising antibody levels were found to be lower than in matched, uninfected control workers. Other evidence points to a reduced antibody response to vaccination among solid organ transplant recipients. With an apparent lack of data on how underlying chronic health conditions impact on antibody response, a team from the Department of Medicine, National Jewish Health, Denver, US, decided to examine the real-world antibody response among vaccine recipients with a range of underlying chronic health problems. The team used the National Jewish Health electronic medical records to identify those who were fully vaccinated and had spike IgG antibody readings done at least 14 days after the second dose. These results were considered as either positive or negative and the team used multivariate logistic regression analysis to identify any clinical characteristics associated with negative spike IgG antibody levels.

Findings
The researchers identified 226 patients with a mean age of 62 years (62% female), of whom 66% had been fully vaccinated with BNT162b and the remainder, mRNA-1273. After a mean of 62 days, just over a quarter (26%) of all patients had no detectable levels of COVID-19 antibodies, 47 given BNT162b and 11 mRNA-1273. The proportion testing negative varied considerably depending on the chronic health condition. For example, 14% of those with chronic obstructive pulmonary disease tested negative, 29% with diabetes, 46% with interstitial lung disease and the highest level, at 53%, was found in those with congestive heart failure. Using regression analysis, the authors calculated that the presence of interstitial lung disease was a significant risk factor for a negative antibody response (odds ratio, OR = 0.21, 95% CI 0.08–0.56), as was congestive heart failure (OR = 0.26) and the use of biologics or Janus kinase inhibitor drugs (OR = 0.17). Interestingly, there was no significant impact from other medicines such as systemic corticosteroids and immunosuppressants such as methotrexate, mycophenolate, azathioprine and tacrolimus.

The authors discussed that though it is widely accepted that no vaccine offers complete protection against infection, their study has raised concerns that among a proportion of patients with chronic health conditions, the absence of detectable antibodies could indicate that such individuals have no protection and are therefore at risk of breakthrough infections. Nevertheless, they also noted that to date, there have been no infections in these patients and that it is possible that an immune response could still be possible. They called for further studies of immunologic response to define ongoing COVID-19 risk in such patients.

Citation
Liao SY et al. Impaired SARS-CoV-2 mRNA vaccine antibody response in chronic medical conditions: a real-world data analysis. MedRxiv 2021