Background: Economic evaluations provide evidence on whether or not digital interventions offer value for money, based on their costs and outcomes relative to the costs and outcomes of alternatives.
Objectives: (1) Evaluate and summarise published economic studies about digital interventions across different technologies, therapies, comparators and mental health conditions; (2) synthesise clinical evidence about digital interventions for an exemplar mental health condition; (3) construct an economic model for the same exemplar mental health condition using the previously synthesised clinical evidence; and (4) consult with stakeholders about how they understand and assess the value of digital interventions.
Methods: We completed four work packages: (1) a systematic review and quality assessment of economic studies about digital interventions; (2) a systematic review and network meta-analysis of randomised controlled trials on digital interventions for generalised anxiety disorder; (3) an economic model and value-of-information analysis on digital interventions for generalised anxiety disorder; and (4) a series of knowledge exchange face-to-face and digital seminars with stakeholders.
Results: In work package 1, we reviewed 76 economic evaluations: 11 economic models and 65 within-trial analyses. Although the results of the studies are not directly comparable because they used different methods, the overall picture suggests that digital interventions are likely to be cost-effective, compared with no intervention and non-therapeutic controls, whereas the value of digital interventions compared with face-to-face therapy or printed manuals is unclear. In work package 2, we carried out two network meta-analyses of 20 randomised controlled trials of digital interventions for generalised anxiety disorder with a total of 2350 participants. The results were used to inform our economic model, but when considered on their own they were inconclusive because of the very wide confidence intervals. In work package 3, our decision-analytic model found that digital interventions for generalised anxiety disorder were associated with lower net monetary benefit than medication and face-to-face therapy, but greater net monetary benefit than non-therapeutic controls and no intervention. Value for money was driven by clinical outcomes rather than by intervention costs, and a value-of-information analysis suggested that uncertainty in the treatment effect had the greatest value (£12.9B). In work package 4, stakeholders identified several areas of benefits and costs of digital interventions that are important to them, including safety, sustainability and reducing waiting times. Four factors may influence their decisions to use digital interventions, other than costs and outcomes: increasing patient choice, reaching underserved populations, enabling continuous care and accepting the 'inevitability of going digital'.
Limitations: There was substantial uncertainty around effect estimates of digital interventions compared with alternatives. This uncertainty was driven by the small number of studies informing most comparisons, the small samples in some of these studies and the studies' high risk of bias.
Conclusions: Digital interventions may offer good value for money as an alternative to 'doing nothing' or 'doing something non-therapeutic' (e.g. monitoring or having a general discussion), but their added value compared with medication, face-to-face therapy and printed manuals is uncertain. Clinical outcomes rather than intervention costs drive 'value for money'.
Future work: There is a need to develop digital interventions that are more effective, rather than just cheaper, than their alternatives.
Study registration: This study is registered as PROSPERO CRD42018105837.
Funding: This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 26, No. 1. See the NIHR Journals Library website for further project information.
Keywords: ANXIETY DISORDERS; COST–BENEFIT ANALYSIS; ECONOMIC MODELS; INTERNET; NETWORK META-ANALYSIS; PROBLEM BEHAVIOUR; PSYCHOTHERAPY; SELF-CARE; SMARTPHONE; SOFTWARE; VIRTUAL REALITY.
Digital interventions are activities accessed via technology platforms (e.g. computers, smartphones and virtual reality) that can improve users’ mental health and reduce addiction problems. To assess whether or not digital interventions offer ‘value for money’, we needed to compare their costs and outcomes with the costs and outcomes of alternatives, such as face-to-face therapy and medication. This was done through economic evaluations. This project consisted of four work packages. In work package 1, we reviewed 76 published economic evaluations of digital interventions for different mental health and addiction problems. We could not directly compare their results because of differences in the methods that were used, but the overall picture suggested that digital interventions could offer good value for money as an alternative to ‘doing nothing’ or simply monitoring someone or giving them general information. The picture was unclear when digital interventions were compared with face-to-face therapy. In work package 2, we pooled research studies that evaluated the outcomes of digital interventions in reducing anxiety and worry; the results were inconclusive because we were uncertain about the differences in outcomes between digital interventions and alternatives. In work package 3, an economic model suggested that value for money in digital interventions is driven by how good they are and not by how much they cost. In work package 4, we presented our methods and results to service users, mental health professionals and researchers who wanted to know more about the value of digital interventions for specific groups (e.g. children and older adults) and for outcomes other than reducing symptoms (e.g. reducing waiting times for treatment and improving attendance for therapy). Finally, the stakeholders highlighted four factors that may influence their decisions to use digital interventions, other than costs and outcomes: increasing choice, reaching underserved populations, enabling continuous care and accepting the ‘inevitability of going digital’.