Aversion to pragmatic randomised controlled trials: three survey experiments with clinicians and laypeople in the USA

Abstract

Objectives: Pragmatic randomised controlled trials (pRCTs) are essential for determining the real-world safety and effectiveness of healthcare interventions. However, both laypeople and clinicians often demonstrate experiment aversion: preferring to implement either of two interventions for everyone rather than comparing them to determine which is best. We studied whether clinician and layperson views of pRCTs for COVID-19, as well as non-COVID-19, interventions became more positive during the pandemic, which increased both the urgency and public discussion of pRCTs.

Design: Randomised survey experiments.

Setting: Geisinger, a network of hospitals and clinics in central and northeastern Pennsylvania, USA; Amazon Mechanical Turk, a research participant platform used to recruit online participants residing across the USA. Data were collected between August 2020 and February 2021.

Participants: 2149 clinicians (the types of people who conduct or make decisions about conducting pRCTs) and 2909 laypeople (the types of people who are included in pRCTs as patients). The clinician sample was primarily female (81%), comprised doctors (15%), physician assistants (9%), registered nurses (54%) and other medical professionals, including other nurses, genetic counsellors and medical students (23%), and the majority of clinicians (62%) had more than 10 years of experience. The layperson sample ranges in age from 18 to 88 years old (mean=38, SD=13) and the majority were white (75%) and female (56%).

Outcome measures: Participants read vignettes in which a hypothetical decision-maker who sought to improve health could choose to implement intervention A for all, implement intervention B for all, or experimentally compare A and B and implement the superior intervention. Participants rated and ranked the appropriateness of each decision. Experiment aversion was defined as the degree to which a participant rated the experiment below their lowest-rated intervention.

Results: In a survey of laypeople administered during the pandemic, we found significant aversion to experiments involving catheterisation checklists and hypertension drugs unrelated to the treatment of COVID-19 (Cohen's d=0.25-0.46, p<0.001). Similarly, among both laypeople and clinicians, we found significant aversion to most (comparing different checklist, proning and mask protocols; Cohen's d=0.17-0.56, p<0.001) but not all (comparing school reopening protocols; Cohen's d=0.03, p=0.64) non-pharmaceutical COVID-19 experiments. Interestingly, we found the lowest experiment aversion to pharmaceutical COVID-19 experiments (comparing new drugs and new vaccine protocols for treating the novel coronavirus; Cohen's d=0.04-0.12, p=0.12-0.55). Across all vignettes and samples, 28%-57% of participants expressed experiment aversion, whereas only 6%-35% expressed experiment appreciation by rating the trial higher than their highest-rated intervention.

Conclusions: Advancing evidence-based medicine through pRCTs will require anticipating and addressing experiment aversion among patients and healthcare professionals.

Study registration: http://osf.io/6p5c7/.

Keywords: COVID-19; ETHICS (see Medical Ethics); PUBLIC HEALTH; Quality Improvement; Randomized Controlled Trial; Surveys and Questionnaires.