The Fragility of Statistically Significant Findings in Pediatric Critical Care Randomized Controlled Trials

Pediatr Crit Care Med. 2019 Jun;20(6):e258-e262. doi: 10.1097/PCC.0000000000001922.

Abstract

Objectives: The Fragility Index measures the number of events on which the statistical significance of a result depends and has been suggested as an adjunct statistical assessment for interpretation of trial results. This study aimed to assess the robustness of statistically significant results from pediatric critical care randomized controlled trials with dichotomous outcomes.

Data sources: A previously published scoping review of pediatric critical care randomized controlled trials (www.PICUtrials.net).

Study selection: A total of 342 trials were screened for inclusion. After applying inclusion/exclusion criteria, 43 fulfilled eligibility criteria and were included in the analysis.

Data extraction: Calculation of Fragility Index for trials reporting a statistically significant dichotomous outcome, and analysis of the relationship between trial characteristics and Fragility Index.

Data synthesis: The median Fragility Index was 2 (interquartile range, 1-6). The median sample size was 98 (interquartile range, 50-148) and sample size demonstrated a strong correlation with the Fragility Index (r = 0.729; n = 43; p < 0.001). The median number of outcome events was 8 (interquartile range, 4-15) and the total number of outcome events also showed a strong correlation with the Fragility Index (r = 0.728; n = 43; p < 0.001).

Conclusions: Results from pediatric critical care randomized controlled trials with dichotomous outcomes reporting statistically significant findings often hinge on a small number of outcome events. Clinicians should exercise caution when interpreting results of trials with a low Fragility Index.

Publication types

  • Systematic Review

MeSH terms

  • Critical Care
  • Data Interpretation, Statistical*
  • Humans
  • Intensive Care Units, Pediatric / statistics & numerical data*
  • Randomized Controlled Trials as Topic / statistics & numerical data*
  • Research Design