We demonstrate a significant reduction of stimulated Brillouin scattering by polarization smoothing in large-scale high-temperature hohlraum plasma conditions where filamentation is measured to be negligible. The stimulated Brillouin scattering experimental threshold (defined as the intensity at which 5% of the incident light is backscattered) is measured to increase by a factor of 1.7+/-0.2 when polarization smoothing is applied. An analytical model relevant to inertial confinement fusion plasma conditions shows that the measured reduction in backscatter with polarization smoothing results from the random spatial variation in polarization of the laser beam, not from the reduction in beam contrast.