Uncovering the physical contents of the nontrivial topology of quantum states is a critical problem in condensed matter physics. Here, we study the topological circular dichroism in chiral semimetals using linear response theory and first-principles calculations. We show that, when the low-energy spectrum respects emergent SO(3) rotational symmetry, topological circular dichroism is forbidden for Weyl fermions, and thus is unique to chiral multifold fermions. This is a result of the selection rule that is imposed by the emergent symmetry under the combination of particle-hole conjugation and spatial inversion. Using first-principles calculations, we predict that topological circular dichroism occurs in CoSi for photon energy below about 0.2 eV. Our Letter demonstrates the existence of a response property of unconventional fermions that is fundamentally different from the response of Dirac and Weyl fermions, motivating further study to uncover other unique responses.