The generation of frequency combs in the mid-infrared (MIR) spectral range by quantum cascade lasers (QCLs) has the potential for revolutionizing dual-comb multi-heterodyne spectroscopy in the molecular fingerprint region. However, in contrast to frequency combs based on passively mode-locked ultrafast lasers, their operation relies on a completely different mechanism resulting from a four-wave mixing process occurring in the semiconductor gain medium that locks the modes together. As a result, these lasers do not emit pulses and no direct self-referencing of a QCL comb spectrum has been achieved so far. Here, we present a detailed frequency noise characterization of a MIR QCL frequency comb operating at a wavelength of 8 µm with a mode spacing of ∼7.4 GHz. Using a beat measurement with a narrow-linewidth single-mode QCL in combination with a dedicated electrical scheme, we measured the frequency noise properties of an optical mode of the QCL comb, and indirectly of its offset frequency for the first time, without detecting it by the standard approach of nonlinear interferometry applied to ultrafast mode-locked lasers. In addition, we also separately measured the noise of the comb mode spacing extracted electrically from the QCL. We observed a strong anti-correlation between the frequency fluctuations of the offset frequency and mode spacing, leading to optical modes with a linewidth slightly below 1 MHz in the free-running QCL comb (at 1-s integration time), which is narrower than the individual contributions of the offset frequency and mode spacing that are at least 2 MHz each.