A rapid mixed-methods evaluation of remote home monitoring models during the COVID-19 pandemic in England

Health Soc Care Deliv Res. 2023 Jul;11(13):1-151. doi: 10.3310/FVQW4410.

Abstract

Background: Remote home monitoring services were developed and implemented for patients with COVID-19 during the pandemic. Patients monitored blood oxygen saturation and other readings (e.g. temperature) at home and were escalated as necessary.

Objective: To evaluate effectiveness, costs, implementation, and staff and patient experiences (including disparities and mode) of COVID-19 remote home monitoring services in England during the COVID-19 pandemic (waves 1 and 2).

Methods: A rapid mixed-methods evaluation, conducted in two phases. Phase 1 (July-August 2020) comprised a rapid systematic review, implementation and economic analysis study (in eight sites). Phase 2 (January-June 2021) comprised a large-scale, multisite, mixed-methods study of effectiveness, costs, implementation and patient/staff experience, using national data sets, surveys (28 sites) and interviews (17 sites).

Results: Phase 1 Findings from the review and empirical study indicated that these services have been implemented worldwide and vary substantially. Empirical findings highlighted that communication, appropriate information and multiple modes of monitoring facilitated implementation; barriers included unclear referral processes, workforce availability and lack of administrative support. Phase 2 We received surveys from 292 staff (39% response rate) and 1069 patients/carers (18% response rate). We conducted interviews with 58 staff, 62 patients/carers and 5 national leads. Despite national roll-out, enrolment to services was lower than expected (average enrolment across 37 clinical commissioning groups judged to have completed data was 8.7%). There was large variability in implementation of services, influenced by patient (e.g. local population needs), workforce (e.g. workload), organisational (e.g. collaboration) and resource (e.g. software) factors. We found that for every 10% increase in enrolment to the programme, mortality was reduced by 2% (95% confidence interval: 4% reduction to 1% increase), admissions increased by 3% (-1% to 7%), in-hospital mortality fell by 3% (-8% to 3%) and lengths of stay increased by 1.8% (-1.2% to 4.9%). None of these results are statistically significant. We found slightly longer hospital lengths of stay associated with virtual ward services (adjusted incidence rate ratio 1.05, 95% confidence interval 1.01 to 1.09), and no statistically significant impact on subsequent COVID-19 readmissions (adjusted odds ratio 0.95, 95% confidence interval 0.89 to 1.02). Low patient enrolment rates and incomplete data may have affected chances of detecting possible impact. The mean running cost per patient varied for different types of service and mode; and was driven by the number and grade of staff. Staff, patients and carers generally reported positive experiences of services. Services were easy to deliver but staff needed additional training. Staff knowledge/confidence, NHS resources/workload, dynamics between multidisciplinary team members and patients' engagement with the service (e.g. using the oximeter to record and submit readings) influenced delivery. Patients and carers felt services and human contact received reassured them and were easy to engage with. Engagement was conditional on patient, support, resource and service factors. Many sites designed services to suit the needs of their local population. Despite adaptations, disparities were reported across some patient groups. For example, older adults and patients from ethnic minorities reported more difficulties engaging with the service. Tech-enabled models helped to manage large patient groups but did not completely replace phone calls.

Limitations: Limitations included data completeness, inability to link data on service use to outcomes at a patient level, low survey response rates and under-representation of some patient groups.

Future work: Further research should consider the long-term impact and cost-effectiveness of these services and the appropriateness of different models for different groups of patients.

Conclusions: We were not able to find quantitative evidence that COVID-19 remote home monitoring services have been effective. However, low enrolment rates, incomplete data and varied implementation reduced our chances of detecting any impact that may have existed. While services were viewed positively by staff and patients, barriers to implementation, delivery and engagement should be considered.

Study registration: This study is registered with the ISRCTN (14962466).

Funding: This project was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme (RSET: 16/138/17; BRACE: 16/138/31) and NHSEI and will be published in full in Health and Social Care Delivery Research; Vol. 11, No. 13. See the NIHR Journals Library website for further project information. The views expressed in this publication are those of the authors and not necessarily those of the National Institute for Health and Care Research or the Department of Health and Social Care.

Plain language summary

COVID-19 patients can experience very low oxygen levels, without feeling breathless. Patients may not realise there is a problem until they become extremely unwell, risking being admitted to hospital too late. To address this, COVID-19 remote home monitoring services were developed and later rolled out across England. Patients monitored oxygen levels at home using an ‘oximeter’ (a small device which clips on to your finger) and sent these readings to providers via phone or technology (e.g. an app). Patients could access further care if needed. We did not know whether these services worked, or what people felt about them. • How services were set up and used in England. • Whether services work (e.g. by reducing deaths and length of hospital stay). • How much they cost. • What patients, carers and staff think about these services (including differences between groups and telephone vs. technology). We looked at available existing evidence and collected data from eight services operating in the first wave of the pandemic. During the second wave of the pandemic, we used data available at a national level and conducted surveys (28 sites) and interviews (17 sites) with staff, patients and individuals involved in developing/leading services nationally. These services have been used worldwide, but they vary considerably. We found many things that help these services to be used (e.g. good communication) but also things that get in the way (e.g. unclear referrals). Our findings did not show that services reduce deaths or time in hospital. But these findings are limited by a lack of data. Staff and patients liked these services, but we found some barriers to delivering and using the service. Some groups found services harder to use (e.g. older patients, those with disabilities and ethnic minorities). Using technology helped with large patient groups, but it did not completely replace phone calls. Better information is needed to know whether these services work. Staff and patients liked these services. However, improvements may make them easier to deliver and use (e.g. further staff training and giving additional support to patients who need it).

MeSH terms

  • Academies and Institutes
  • Aged
  • Braces
  • COVID-19* / epidemiology
  • England / epidemiology
  • Humans
  • Pandemics
  • Systematic Reviews as Topic