The neutron spectrum produced by deuterium-tritium (DT) inertial confinement fusion implosions contains a wealth of information about implosion performance including the DT yield, ion-temperature, and areal-density. The Magnetic Recoil Spectrometer (MRS) has been used at both the OMEGA laser facility and the National Ignition Facility (NIF) to measure the absolute neutron spectrum from 3 to 30 MeV at OMEGA and 3 to 36 MeV at the NIF. These measurements have been used to diagnose the performance of cryogenic target implosions to unprecedented accuracy. Interpretation of MRS data requires a detailed understanding of the MRS response and background. This paper describes ab initio characterization of the system involving Monte Carlo simulations of the MRS response in addition to the commission experiments for in situ calibration of the systems on OMEGA and the NIF.