Two experiments examined the effects of visual similarity on short-term recall for where and when in the visual spatial domain. A series of squares of similar or dissimilar colours were serially presented at various locations on the screen. At recall, all coloured squares were simultaneously presented in a random order at the bottom of the screen, and the locations used for presentation were indicated by white squares. Participants were asked to place the colours at their appropriate location in their presentation order. Performance for location (where) and order (when) was assessed separately. Results revealed that similarity severely hinders both memory for what was where and memory for what was when, under quiet and articulatory suppression conditions. These results provide further evidence that similarity has a major impact on processing relational information in memory.