Using electron bunches generated by laser wakefield acceleration as a probe, the temporal evolution of magnetic fields generated by a 4 × 10(19) W/cm(2) ultrashort (30 fs) laser pulse focused on solid density targets is studied experimentally. Magnetic field strengths of order B(0) ~ 10(4) T are observed expanding at close to the speed of light from the interaction point of a high-contrast laser pulse with a 10-μm-thick aluminum foil to a maximum diameter of ~1 mm. The field dynamics are shown to agree with particle-in-cell simulations.