Chronic olfactory disorders, including the complete loss of the sense of smell (anosmia), are common. Using voxel-based morphometry (VBM) in magnetic resonance imaging (MRI), structural changes in the cerebral gray matter (GM) of a group of patients with anosmia compared with a normosmic, healthy control group were evaluated. Patients with anosmia presented a significant decrease of GM volume mainly in the nucleus accumbens with adjacent subcallosal gyrus, in the medial prefrontal cortex (MPC) including the middle and anterior cingulate cortices, and in the dorsolateral prefrontal cortex (dlPFC). These areas are part of the limbic loop of the basal ganglia and except the dlPFC secondary olfactory areas. They also play an important role in many neurological diseases. Furthermore, volume decreases in smaller areas like the piriform cortex, insular cortex, orbitofrontal cortex, hippocampus, parahippocampal gyrus, supramarginal gyrus, and cerebellum could be seen. Longer disease duration was associated with a stronger atrophy in the described areas. No local increases in the GM volume could be observed. A comparison with results of an additionally executed functional MRI study on olfaction in healthy subjects was performed to evaluate the significance of the observed atrophy areas in cerebral olfactory processing. To our knowledge, this is the first study on persisting structural changes in cortical GM volume after complete olfactory loss.