Phase 3 trials are the mainstay of drug development across medicine but have often not met expectations set by preceding phase 2 studies. A systematic meta-analysis evaluated all randomized controlled, double-blind trials investigating targeted disease-modifying anti-rheumatic drugs in rheumatoid and psoriatic arthritis. Primary outcomes of American College of Rheumatology (ACR) 20 responses were compared by mixed-model logistic regression, including exploration of potential determinants of efficacy overestimation. In rheumatoid arthritis, phase 2 trial outcomes systematically overestimated subsequent phase 3 results (odds ratio comparing ACR20 in phase 2 versus phase 3: 1.39, 95% confidence interval: 1.25-1.57, P < 0.001). Data for psoriatic arthritis trials were similar, but not statistically significant (odds ratio comparing ACR20 in phase 2 versus phase 3: 1.35, 95% confidence interval: 0.94-1.94, P = 0.09). Differences in inclusion criteria largely explained the observed differences in efficacy findings. Our findings have implications for all stakeholders in new therapeutic development and testing, as well as potential ethical implications.