Duplication of genes, genomes, or morphological structures (or some combination of these) has long been thought to facilitate evolutionary change. Here we focus on studies of the teleost fishes to consider the conceptual similarities in the evolutionary potential of these three different kinds of duplication events. We review recent data that have confirmed the occurrence of a whole-genome duplication event in the ray-finned fish lineage, and discuss whether this event may have fuelled the radiation of teleost fishes. We then consider the fates of individual duplicated genes, from both a theoretical and an experimental viewpoint, focusing on our studies of teleost Hox genes and their functions in patterning the segmented hindbrain. Finally, we consider the duplication of morphological structures, once again drawing on our experimental studies of the hindbrain, which have revealed that experimentally induced duplicated neurons can produce functionally redundant neural circuits. We posit that the availability of duplicated material, independent of its nature, can lead to functional redundancy, which in turn enables evolutionary change.