Call to Improve the Quality of Prediction Tools for Intrahepatic Cholangiocarcinoma Resection: A Critical Appraisal, Systematic Review, and External Validation Study

Woo Jin Choi; Richard Walker; Luckshi Rajendran; Owen Jones; Annie Gravely; Marina Englesakis; Steven Gallinger; Gideon Hirschfield; Bettina Hansen; Gonzalo Sapisochin

doi:10.1097/AS9.0000000000000328

Call to Improve the Quality of Prediction Tools for Intrahepatic Cholangiocarcinoma Resection: A Critical Appraisal, Systematic Review, and External Validation Study

Ann Surg Open. 2023 Sep 1;4(3):e328. doi: 10.1097/AS9.0000000000000328. eCollection 2023 Sep.

Authors

Woo Jin Choi^{1

2}, Richard Walker^{1

2}, Luckshi Rajendran¹, Owen Jones³, Annie Gravely³, Marina Englesakis⁴, Steven Gallinger^{1

3}, Gideon Hirschfield^{2

5

6}, Bettina Hansen^{2

6

7}, Gonzalo Sapisochin^{1

2

3}

Affiliations

¹ From the Department of Surgery, University of Toronto, Toronto, Ontario, Canada.
² Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada.
³ University Health Network, HPB Surgical Oncology, Toronto, Ontario, Canada.
⁴ Library and Information Services, University Health Network, Toronto, Canada.
⁵ Department of Medicine, University of Toronto, Toronto, Ontario, Canada.
⁶ Toronto Centre for Liver Disease, Toronto General Hospital, University Health Network, Toronto, Canada.
⁷ Department of Epidemiology & Biostatistics, Erasmus MC, Rotterdam, the Netherlands.

Abstract

Objective: To conduct a systematic review, critical appraisal, and external validation of survival prediction tools for patients undergoing intrahepatic cholangiocarcinoma (iCCA) resection.

Summary background data: Despite the development of several survival prediction tools in recent years for patients undergoing iCCA resections, there is a lack of critical appraisal and external validation of these models.

Methods: We conducted a systematic review and critical appraisal of survival and recurrence prediction models for patients undergoing curative-intent iCCA resections. Studies were evaluated based on their model design, risk of bias, reporting, performance, and validation results. We identified the best model and externally validated it using our institution's data.

Results: This review included a total of 31 studies, consisting of 26 studies with original prediction tools and 5 studies that only conducted external validations. Among the 26, 54% of the studies conducted internal validations, 46% conducted external validations, and only 1 study scored a low risk of bias. Harrell's C-statistics ranged from 0.67 to 0.76 for internal validation and from 0.64 to 0.75 for external validation. Only 81% of the studies reported model calibration. Our external validation of the best model (Intrahepatic Cholangiocarcinoma [ICC]-Metroticket) estimated Harrell's and Uno's C-statistics of 0.67 (95% CI: 0.56-0.77) and Uno's time-dependent area under the receiver operating characteristic curve (AUC) of 0.71 (95% CI: 0.53-0.88), with a Brier score of 0.20 (95% CI: 0.15-0.26) and good calibration plots.

Conclusions: Many prediction models have been published in recent years, but their quality remains poor, and minimal methodological quality improvement has been observed. The ICC-Metroticket was selected as the best model (Uno's time-dependent AUC of 0.71) for 5-year overall survival prediction in patients undergoing curative-intent iCCA resection.

Keywords: cholangiocarcinoma; prediction; resection; tool; validation.