The membrane-bound enzyme cytochrome c oxidase is responsible for cell respiration in aerobic organisms and conserves free energy from O2 reduction into an electrochemical proton gradient by coupling the redox reaction to proton-pumping across the membrane. O2 reduction produces water at the bimetallic heme a3/CuB active site next to a hydrophobic cavity deep within the membrane. Water molecules in this cavity have been suggested to play an important role in the proton-pumping mechanism. Here, we show by molecular dynamics simulations that the conserved arginine/heme a3 delta-propionate ion pair provides a gate, which exhibits reversible thermal opening that is governed by the redox state and the water molecules in the cavity. An important role of this gate in the proton-pumping mechanism is supported by site-directed mutagenesis experiments. Transport of the product water out of the enzyme must be rigidly controlled to prevent water-mediated proton leaks that could compromise the proton-pumping function. Exit of product water is observed through the same arginine/propionate gate, which provides an explanation for the observed extraordinary spatial specificity of water expulsion from the enzyme.