Experiments in animals leave no doubt that androgens, including testosterone, produced by the testes in fetal and/or neonatal life act on the brain to induce sex differences in neural structure and function. In human beings, there is evidence supporting a female superiority in the ability to read nonverbal signals, specific language-related skills, and theory of mind. Even more striking than the sex differences seen in the typical population is the elevated occurrence of social and communicative difficulties in human males. One such condition, autism, occurs four times more frequently in boys than in girls. Recently, a novel theory known as the "extreme male brain" has been proposed. It suggests that the behaviors seen in autism are an exaggeration of typical sex differences and that exposure to high levels of prenatal testosterone might be a risk factor. In this article, we argue that prenatal and neonatal testosterone exposures are strong candidates for having a causal role in sexual dimorphism in human behavior, including social development, and as risk factors for conditions characterized by social impairments, particularly autism spectrum conditions.