A multivariate model for successful publication of intensive care medicine randomized controlled trials in the highest impact factor journals: the SCOTI score

Joris Pensier; Audrey De Jong; Gerald Chanques; Emmanuel Futier; Elie Azoulay; Nicolas Molinari; Samir Jaber

doi:10.1186/s13613-021-00954-x

A multivariate model for successful publication of intensive care medicine randomized controlled trials in the highest impact factor journals: the SCOTI score

Ann Intensive Care. 2021 Nov 27;11(1):165. doi: 10.1186/s13613-021-00954-x.

Authors

Joris Pensier¹, Audrey De Jong¹, Gerald Chanques¹, Emmanuel Futier², Elie Azoulay³, Nicolas Molinari^{4

5}, Samir Jaber^{6

7}

Affiliations

¹ Department of Anesthesia and Intensive Care Unit, Regional University Hospital of Montpellier, St-Eloi Hospital, University of Montpellier, PhyMedExp, INSERM U1046, CNRS UMR, 9214 CEDEX 5, Montpellier, France.
² Department of Peri-Operative Medicine, CHU Clermont-Ferrand, 63000, Clermont-Ferrand, France.
³ Médecine Intensive et Réanimation, Groupe FAMIREA, Hôpital Saint-Louis, Université de Paris, Paris, France.
⁴ IDESP, INSERM, Univ Montpellier, CHU Montpellier, Montpellier, France.
⁵ Universite de Montpellier, Montpellier, Languedoc-Roussillon, France.
⁶ Department of Anesthesia and Intensive Care Unit, Regional University Hospital of Montpellier, St-Eloi Hospital, University of Montpellier, PhyMedExp, INSERM U1046, CNRS UMR, 9214 CEDEX 5, Montpellier, France. s-jaber@chu-montpellier.fr.
⁷ Département d'Anesthésie Réanimation B (DAR B), 80 Avenue Augustin Fliche, 34295, Montpellier, France. s-jaber@chu-montpellier.fr.

Abstract

Background: Critical care randomized controlled trials (RCTs) are often published in high-impact journals, whether general journals [the New England Journal of Medicine (NEJM), The Lancet, the Journal of the American Medical Association (JAMA)] or critical care journals [Intensive Care Medicine (ICM), the American Journal of Respiratory and Critical Care Medicine (AJRCCM), Critical Care Medicine (CCM)]. As rejection occurs in up to 97% of cases, it might be appropriate to assess pre-submission probability of being published. The objective of this study was to develop and internally validate a simplified score predicting whether an ongoing trial stands a chance of being published in high-impact general journals.

Methods: A cohort of critical care RCTs published between 1999 and 2018 in the three highest impact medical journals (NEJM, The Lancet, JAMA) or the three highest impact critical care journals (ICM, AJRCCM, CCM) was split into two samples (derivation cohort, validation cohort) to develop and internally validate the simplified score. Primary outcome was journal of publication assessed as high-impact general journal (NEJM, The Lancet, JAMA) or critical care journal (ICM, AJRCCM, CCM).

Results: A total of 968 critical care RCTs were included in the predictive cohort and split into a derivation cohort (n = 510) and a validation cohort (n = 458). In the derivation cohort, the sample size (P value < 0.001), the number of centers involved (P value = 0.01), mortality as primary outcome (P value = 0.002) or a composite item including mortality as primary outcome (P value = 0.004), and topic [ventilation (P value < 0.001) or miscellaneous (P value < 0.001)] were independent factors predictive of publication in high-impact general journals, compared to high-impact critical care journals. The SCOTI score (Sample size, Centers, Outcome, Topic, and International score) was developed with an area under the ROC curve of 0.84 (95% Confidence Interval, 0.80-0.88) in validation by split sample.

Conclusions: The SCOTI score, developed and validated by split sample, accurately predicts the chances of a critical care RCT being published in high-impact general journals, compared to high-impact critical care journals.

Keywords: Critical care; Critically ill patients; Intensive care medicine; Intensive care unit; Precision medicine.