We develop a theory of Coulomb drag due to momentum transfer between graphene layers in a strong magnetic field. The theory is intended to apply in systems with disorder that is weak compared to Landau level separation, so that Landau level mixing is weak but strong compared to correlation energies within a single Landau level, so that fractional quantum Hall physics is not relevant. We find that, in contrast to the zero-field limit, the longitudinal magneto-Coulomb drag is finite and, in fact, attains a maximum at the simultaneous charge neutrality point (CNP) of both layers. Our theory also predicts a sizable Hall drag resistivity at densities away from the CNP.