Anti-Motion Remote Measurement of Heart Rate Based on Region Proposal Generation and Multi-Scale ROI Fusion

• Published in: IEEE transactions on instrumentation and measurement Vol. 71; pp. 1 - 13
• Main Authors: Zhao, Changchen, Zhou, Menghao, Han, Weiran, Feng, Yuanjing
• Format: Journal Article
• Language: English
• Published: New York IEEE 2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Aspect ratio; Blood volume; Bounding box; Boxes; Datasets; Faces; facial video; Feature extraction; Head; Head movement; Heart rate; heart rate (HR) estimation; Modules; multi-scale region of interest (ROI); Object detection; Perturbation; Pixels; Proposals; remote photoplethysmography (rPPG); Root-mean-square errors; Signal to noise ratio; Skin; Source separation; Tracking errors
• ISSN: 0018-9456; 1557-9662
• DOI: 10.1109/TIM.2022.3169567

Remote detection of human heart rate (HR) in a non-contact manner has become an active research topic in recent years. Most existing works focus on signal separation of raw traces obtained from the facial region of interest (ROI), whereas the ROI itself has a crucial impact on the measurement accuracy. The ROI tracking issues such as not fitting the subject's face, pixel shift when there is head motion, and fluctuation when keeping still cannot be effectively dealt with by existing remote photoplethysmography (rPPG) algorithms, which seriously affects the pulse measurement accuracy. This article addresses these issues by proposing a novel pulse signal extraction framework based on generating region proposals and multi-scale ROIs. The framework consists of two signal selection modules and one signal fusion module. First, the multiple aspect ratio sampling operation is proposed to generate bounding boxes with various aspect ratios to remove background pixels as well as to fit different subjects' faces. Second, the bounding box perturbation strategy is proposed to generate bounding boxes around the selected position to reduce the tracking error caused by head translation and rotation. Finally, multi-scale signal fusion (MSSF) is proposed for fusion signals from multi-scale ROI. Extensive experiments are conducted on two publicly available datasets and one self-collected dataset. The results show that the proposed framework achieves significant performance improvement in a signal-to-noise ratio (SNR), mean absolute error (MAE), and root mean squared error (RMSE). On pulse rate detection dataset (PURE), Université Bourgogne Franche-Comté (UBFC)-rPPG, and Self-rPPG datasets, the SNR has a maximum increase of 2.848 dB [compared with chrominance-based rPPG (CHROM)], 1.690 dB [compared with plane-orthogonal-to-skin (POS)], and 1.806 dB (compared with CHROM), respectively. MAE has a maximum decrease of 1.947 bpm (compared with CHROM), 1.610 bpm [compared with pulse blood volume (PBV)], and 1.577 bpm (compared with CHROM), respectively. Moreover, the proposed framework is independent of existing signal extraction algorithms. They can be easily embedded in the proposed framework.