Using high resolution magnetic resonance imaging data, we examined the interrelationships between eight cerebral lobar volumetric measures via both exploratory and confirmatory factor analyses in a large sample (N = 484) of pediatric twins and singletons. These analyses suggest the presence of strong genetic correlations between cerebral structures, particularly between regions of like tissue type or in spatial proximity. Structural modeling estimated that most of the variance in all structures is associated with highly correlated lobar latent factors, with differences in genetic covariance and heritability driven by a common genetic factor that influenced gray and white matter differently. Reanalysis including total brain volume as a covariate dramatically reduced the total residual variance and disproportionately influenced the additive genetic variance in all regions of interest.