Previous research suggests that, in reaction time (RT) measures of episodic memory retrieval, the unique effects of adult age are relatively small compared to the effects aging shares with more elementary abilities such as perceptual speed. Little is known, however, regarding the mechanisms of perceptual speed. We used diffusion tensor imaging (DTI) to test the hypothesis that white matter integrity, as indexed by fractional anisotropy (FA), serves as one mechanism of perceptual slowing in episodic memory retrieval. Results indicated that declines in FA in the pericallosal frontal region and in the genu of the corpus callosum, but not in other regions, mediated the relationship between perceptual speed and episodic retrieval RT. This relation held, though to a different degree, for both hits and correct rejections. These findings suggest that white matter integrity in prefrontal regions is one mechanism underlying the relation between individual differences in perceptual speed and episodic retrieval.