Diffusion tensor imaging (DTI) has been used to study microstructural white matter alterations in a variety of conditions including normal aging and Alzheimer's disease (AD). White matter hyperintensities (WMH) are common in cognitively healthy elderly as well as in AD and exhibit elevated mean diffusivity (MD) and reduced fractional anisotropy (FA). However, the effect of WMH on statistical analysis of DTI estimates has not been thoroughly studied. In the present study we address this in two ways. First, we investigate the effect of WMH on MD and FA in the dorsal and ventral cingulum, the superior longitudinal fasciculus, and the corticospinal tract, by comparing two matched groups of cognitively healthy elderly (n = 21 + 21) with unequal WMH load. Second, we assess the effects of adjusting for WMH load when comparing MD and FA in prodromal AD subjects (n = 83) to cognitively healthy elderly (n = 132) in the abovementioned white matter tracts. Results showed the WMH in cognitively healthy elderly to have a generally large effect on DTI estimates (Cohen's d = 0.63 to 1.27 for significant differences in MD and -1.06 to -0.69 for FA). These effect sizes were comparable to those of various neurological and psychiatric diseases (Cohen's d = 0.57 to 2.20 for differences in MD and -1.76 to -0.61 for FA). Adjusting for WMH when comparing DTI estimates in prodromal AD subjects to cognitively healthy elderly improved the explanatory power as well as the outcome of the analysis, indicating that some of the differences in MD and FA were largely driven by unequal WMH load between the groups rather than alterations in normal-appearing white matter (NAWM). Thus, our findings suggest that if the purpose of a study is to compare alterations in NAWM between two groups using DTI it may be necessary to adjust the statistical analysis for WMH.