Agreement between the Cochrane risk of bias tool and Physiotherapy Evidence Database (PEDro) scale: A meta-epidemiological study of randomized controlled trials of physical therapy interventions

Anne M Moseley; Prinon Rahman; George A Wells; Joshua R Zadro; Catherine Sherrington; Karine Toupin-April; Lucie Brosseau

doi:10.1371/journal.pone.0222770

Agreement between the Cochrane risk of bias tool and Physiotherapy Evidence Database (PEDro) scale: A meta-epidemiological study of randomized controlled trials of physical therapy interventions

PLoS One. 2019 Sep 19;14(9):e0222770. doi: 10.1371/journal.pone.0222770. eCollection 2019.

Authors

Anne M Moseley¹, Prinon Rahman², George A Wells³, Joshua R Zadro¹, Catherine Sherrington¹, Karine Toupin-April^{4

5

6}, Lucie Brosseau²

Affiliations

¹ Institute for Musculoskeletal Health, Sydney School of Public Health, Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia.
² Physiotherapy Program, School of Rehabilitation Sciences, Faculty of Health Sciences, University of Ottawa, Ottawa, Ontario, Canada.
³ School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada.
⁴ Children's Hospital of Eastern Ontario Research Institute, Ottawa, Ontario, Canada.
⁵ Department of Pediatrics, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada.
⁶ School of Rehabilitation Sciences, Faculty of Health Sciences, University of Ottawa, Ottawa, Ontario, Canada.

Abstract

Background: The Cochrane risk of bias (CROB) tool and Physiotherapy Evidence Database (PEDro) scale are used to evaluate risk of bias of randomized controlled trials. We assessed the level of agreement between the instruments.

Methods: We searched the Cochrane Library to identify trials included in systematic reviews evaluating physical therapy interventions. For trials that met our inclusion criteria (primary reference in Cochrane review, review used CROB (2008 version), indexed in PEDro), CROB items were extracted from the reviews and PEDro items and total score were downloaded from PEDro. Kappa statistics were used to determine the agreement between CROB and PEDro scale items that evaluate similar constructs (e.g., randomization). The total PEDro score was compared to the CROB summary score (% of items met) using an Intraclass Correlation Coefficient. Sensitivity analyses explored the impact of the CROB "unclear" category and variants of CROB blinding items. Kappa statistics were used to determine agreement between different thresholds for "acceptable" risk of bias between CROB and PEDro scale summary scores.

Results: We included 1442 trials from 108 Cochrane reviews. Agreement was "moderate" for three of the six CROB and PEDro scale items that evaluate similar constructs (allocation concealment, participant blinding, assessor blinding; Kappa = 0.479-0.582). Agreement between the summary scores was "poor" (Intraclass Correlation Coefficient = 0.285). Agreement was highest when the CROB "unclear" category was collapsed with "high" and when participant, personnel and assessor blinding were evaluated separately in CROB. Agreement for different thresholds for "acceptable" risk of bias between CROB and PEDro summary scores was, at best, "fair".

Conclusion: There was moderate agreement for half of the PEDro and CROB items that evaluate similar constructs. Interpretation of the CROB "unclear" category and variants of the CROB blinding items substantially influenced agreement. Either instrument can be used to quantify risk of bias, but they can't be used interchangeably.

MeSH terms

Bias*
Epidemiologic Studies
Exercise / physiology*
Exercise Therapy / methods*
Humans
Physical Therapy Modalities
Randomized Controlled Trials as Topic / methods*
Systematic Reviews as Topic

Grants and funding

The authors received no specific funding for this work.