The value of a machine learning algorithm to predict adverse short-term outcome during resuscitation of patients with in-hospital cardiac arrest: a retrospective study

Abstract

Background and importance Guidelines recommend that hospital emergency teams locally validate criteria for termination of cardiopulmonary resuscitation in patients with in-hospital cardiac arrest (IHCA). Objective To determine the value of a machine learning algorithm to predict failure to achieve return of spontaneous circulation (ROSC) and unfavourable functional outcome from IHCA using only data readily available at emergency team arrival. Design Retrospective cohort study. Setting and participants Adults who experienced an IHCA were attended to by the emergency team. Outcome measures and analysis Demographic and clinical data typically available at the arrival of the emergency team were extracted from the institutional IHCA database. In addition, outcome data including the Cerebral Performance Category (CPC) score count at hospital discharge were collected. A model selection procedure for random forests with a hyperparameter search was employed to develop two classification algorithms to predict failure to achieve ROSC and unfavourable (CPC 3-5) functional outcomes. Main results Six hundred thirty patients were included, of which 390 failed to achieve ROSC (61.9%). The final classification model to predict failure to achieve ROSC had an area under the receiver operating characteristic curve of 0.9 [95% confidence interval (CI), 0.89-0.9], a balanced accuracy of 0.77 (95% CI, 0.75-0.79), an F1-score of 0.78 (95% CI, 0.76-0.79), a positive predictive value of 0.88 (0.86-0.91), a negative predictive value of 0.61 (0.6-0.63), a sensitivity of 0.69 (0.66-0.72), and a specificity of 0.84 (0.8-0.88). Five hundred fifty-nine subjects experienced an unfavourable outcome (88.7%). The final classification model to predict unfavourable functional outcomes from IHCA at hospital discharge had an area under the receiver operating characteristic curve of 0.93 (95% CI, 0.92-0.93), a balanced accuracy of 0.59 (95% CI, 0.57-0.61), an F1-score of 0.94 (95% CI, 0.94-0.95), a positive predictive value of 0.91 (0.9-0.91), a negative predictive value of 0.57 (0.48-0.66), a sensitivity of 0.98 (0.97-0.99), and a specificity of 0.2 (0.16-0.24). Conclusion Using data readily available at emergency team arrival, machine learning algorithms had a high predictive power to forecast failure to achieve ROSC and unfavourable functional outcomes from IHCA while cardiopulmonary resuscitation was still ongoing; however, the positive predictive value of both models was not high enough to allow for early termination of resuscitation efforts.