Mammalian sex chromosomes are highly diverged and heteromorphic: a comparatively large and gene-rich X chromosome contrasting with a small, largely heterochromatic and degenerate Y chromosome. Both gonosomes are however uniquely important in male-specific functions such as spermatogenesis. In this review, we examine the evolutionary pressures that have driven the divergence of the sex chromosomes from their ancestral state, and show how these have shaped the gene content of both chromosomes. Their shared history of gene acquisition and loss, differentiation, degeneration and intragenomic warfare has far-reaching consequences for their functionality in spermatogenesis, and may also have potential clinical implications.