We report the development of a broadband rotary joint for high-speed ultrahigh-resolution endoscopic optical coherence tomography (OCT) imaging in the 800 nm spectral range. This rotary joint features a pair of achromatic doublets in order to achieve broadband operation for a 3 dB bandwidth over 150 nm. The measured one-way throughput of the rotary joint is greater than 80%, while the fluctuation of the double-pass coupling efficiency during 360 deg rotation is less than ±5% at a near video-rate speed of 20 revolutions/s (rps). The rotary joint is used in conjunction with a diffractive-optics-based endoscope and 800 nm spectral domain OCT system and achieved an ultrahigh axial resolution of ∼2.4 μm in air. The imaging performance is demonstrated by 3D circumferential imaging of a mouse colon in vivo.