In this paper an automatic atlas-based segmentation algorithm for 4D cardiac MR images is proposed. The algorithm is based on the 4D extension of the expectation maximisation (EM) algorithm. The EM algorithm uses a 4D probabilistic cardiac atlas to estimate the initial model parameters and to integrate a priori information into the classification process. The probabilistic cardiac atlas has been constructed from the manual segmentations of 3D cardiac image sequences of 14 healthy volunteers. It provides space and time-varying probability maps for the left and right ventricles, the myocardium, and background structures such as the liver, stomach, lungs and skin. In addition to using the probabilistic cardiac atlas as a priori information, the segmentation algorithm incorporates spatial and temporal contextual information by using 4D Markov Random Fields. After the classification, the largest connected component of each structure is extracted using a global connectivity filter which improves the results significantly, especially for the myocardium. Validation against manual segmentations and computation of the correlation between manual and automatic segmentation on 249 3D volumes were calculated. We used the 'leave one out' test where the image set to be segmented was not used in the construction of its corresponding atlas. Results show that the procedure can successfully segment the left ventricle (LV) (r = 0.96), myocardium (r = 0.92) and right ventricle (r = 0.92). In addition, 4D images from 10 patients with hypertrophic cardiomyopathy were also manually and automatically segmented yielding a good correlation in the volumes of the LV (r = 0.93) and myocardium (0.94) when the atlas constructed with volunteers is blurred.