Tetragonal CuMnAs is an antiferromagnetic material with favourable properties for applications in spintronics. Using a combination of neutron diffraction and x-ray magnetic linear dichroism, we determine the spin axis and magnetic structure in tetragonal CuMnAs, and reveal the presence of an interfacial uniaxial magnetic anisotropy. From the temperature-dependence of the neutron diffraction intensities, the Néel temperature is shown to be (480 ± 5) K. Ab initio calculations indicate a weak anisotropy in the (ab) plane for bulk crystals, with a large anisotropy energy barrier between in-plane and perpendicular-to-plane directions.