Multi-Level Constrained Intra and Inter Subject Feature Representation for Facial Video Based BVP Signal Measurement

• Published in: IEEE journal of biomedical and health informatics Vol. 27; no. 8; pp. 3948 - 3957
• Main Authors: Li, Bin, Zhang, Wei, Fu, Hong, Liu, Hao, Xu, Feng
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.08.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Biomedical measurement; Blood volume; Constraints; Face; Faces; Feature extraction; Global-local temporal fusion; Heart Rate; Humans; Kernels; Monitoring, Physiologic; remote health monitoring; Remote monitoring; Representation learning; Representations; Semantics; Signal measurement; Skin; Temporal variations; Video sequences
• ISSN: 2168-2194; 2168-2208; 2168-2208
• DOI: 10.1109/JBHI.2023.3273557

Facial video-based blood volume pulse (BVP) signal measurement holds great potential for remote health monitoring, while existing methods have issues with convolutional kernel perceptual field constraints. This article proposes an end-to-end multi-level constrained spatiotemporal representation structure for facial video-based BVP signal measurement. First, an intra- and inter-subject feature representation is proposed to strengthen the BVP-related features generation at high, semantic, and shallow levels, respectively. Second, the global-local association is presented to enhance BVP signal period pattern learning, and the global temporal features are introduced into the local spatial convolution of each frame by adaptive kernel weights. Finally, the multi-dimensional fused features are mapped to one-dimensional BVP signals by the task-oriented signal estimator. The experimental results on the publicly available MMSE-HR dataset demonstrate that the proposed structure overperforms state-of-the-art methods (e.g., AutoHR) in BVP signal measurement, with a 20% and 40% reduction in mean absolute error and root mean squared error, respectively. The proposed structure would be a powerful tool for telemedical and non-contact heart health monitoring.