Background: Medical education programs need outcome measurements to promote curriculum improvement and to help meet accreditation standards.
Purpose: Determine the added value of qualitative comments written by program directors (PDs) in response to a survey concerning first postgraduate year (PGY-1) graduates. We hypothesized that these comments would serve as an additional outcome measurement for our graduates, adding information not readily captured in numeric data.
Methods: PD evaluation form surveys from 1993-2002. All qualitative comments offered in response to free text questions were coded and compared with numeric ratings.
Results: A total of 1,247 surveys were included (80% response rate). Comments about specific graduates were coded as positive, negative, or neutral and were categorized into themes. Inter-rater reliability was high (kappa= 0.82). Compared with 4% of graduates who received one or more numeric ratings of less than satisfactory, 7% had one or more qualitative phrases classified as negative.
Conclusions: Qualitative comments can serve as a useful outcome measurement.