The aim of this study was to explore local brain atrophy of patients with alcohol dependence using a voxel-based analysis of magnetic resonance images and to investigate the relationship of those atrophic regions with drinking history and neuropsychological performances. Statistical parametric mapping was applied for the global and regional comparison of segmented gray matter and white matter images from 20 patients with alcohol dependence and with those from 20 controls. The Rey auditory-verbal learning test, Rey-Osterrieth complex figure test, Stroop test, trail-making test, and Wisconsin card sorting test were conducted as neuropsychological evaluations. There was a significant decrease in both gray matter and white matter globally in alcohol dependence. Bilateral parahippocampal white matter areas were reduced in particular. Perseverative responses and perseverative errors in the Wisconsin card sorting test had significant correlation with the decrease of gray matter decrease including the left superior temporal gyri and right postcentral region. The psychological performance measures correlated with gray matter rather than white matter, whereas right temporal white matter correlated with drinking amount for last 4 weeks. This may imply that alcohol consumption in heavy amounts damages both gray matter and white matter, and gray matter atrophy mainly leads to cognitive impairment, whereas white matter is related to drinking history.