Objectives: To determine the impact of introducing patient reported outcome measures (PROMs) on the selection of patients for surgery and on outcome for four elective operations (hip replacement, knee replacement, varicose vein surgery and groin hernia repair).
Methods: Patient-level data from the National PROMs programme for England from April 2009 to March 2012 were analysed to determine the extent of change in mean preoperative scores and mean adjusted postoperative scores using disease-specific and generic PROMs assessed using standardized effect sizes (SESs). Variation between providers was determined by intra-class correlation coefficients for each time period. Consistency in outlier ratings was tested using weighted Kappa statistics.
Results: There was little apparent impact. Preoperative severity increased slightly for two procedures only: varicose vein surgery (SES disease-specific PROM 0.10; generic PROM -0.07) and to a lesser extent for hip replacement (SES disease-specific PROM -0.03). There was little inter-provider variation and it did not change significantly over time. There were also slight improvements in outcomes for hip and knee replacement (SES for disease-specific and generic PROMs 0.03) though not for hernia repair and a slight worsening for varicose vein surgery. The extent of variation in performance between providers was unchanged. The proportion of providers deemed to be outliers did not change over time. There was only moderate consistency in those providers deemed to be outliers for hip and knee replacement (Kappa 0.31-0.47) and it was even weaker for the other two procedures. Although 35% of providers of hip replacement were outliers in at least one year, only 6% were consistently outliers. Such inconsistency may be partly due to regression to the mean.
Conclusions: The minimal impact that the routine use and feedback of PROMs had on provider behaviour during the initial years suggests that more attention needs to be paid to how results are communicated and to the provision of advice as to what action may be taken.
Keywords: health services; quality improvement; surgery.