Breathing rate (BR) is a key physiological parameter used in a range of clinical settings. Despite its diagnostic and prognostic value, it is still widely measured by counting breaths manually. A plethora of algorithms have been proposed to estimate BR from the electrocardiogram (ECG) and pulse oximetry (photoplethysmogram, PPG) signals. These BR algorithms provide opportunity for automated, electronic, and unobtrusive measurement of BR in both healthcare and fitness monitoring. This paper presents a review of the literature on BR estimation from the ECG and PPG. First, the structure of BR algorithms and the mathematical techniques used at each stage are described. Second, the experimental methodologies that have been used to assess the performance of BR algorithms are reviewed, and a methodological framework for the assessment of BR algorithms is presented. Third, we outline the most pressing directions for future research, including the steps required to use BR algorithms in wearable sensors, remote video monitoring, and clinical practice.