This paper proposes an efficient method for the joint localization of sources and estimation of the covariance of their signals. In practice, such an estimation is useful to study correlated sources existing, for instance, in the presence of spatially distributed sources or reflections, but is confronted with the challenge of computational complexity due to a large number of required estimates. The proposed method is called covariance matrix fitting by orthogonal least squares. It is based on a greedy dictionary based approach exploiting the orthogonal least squares algorithm in order to reduce the computational complexity of the estimation. Compared to existing methods for sources correlation matrix estimation, its lower computational complexity allows one to deal with high dimensional problems (i.e., fine discretization of the source space) and to explore large regions of possible sources positions. As shown by numerical results, it is more accurate than existing methods and does not require the tuning of any regularization parameter. Experiments in an anechoic chamber involving correlated sources or reflectors show the ability of the method to locate and identify physical and mirror sources as well.