Remote photoplethysmography with anti-interference based on spatio-temporal feature enhancement

• Published in: Signal, image and video processing Vol. 19; no. 5
• Main Authors: Shao, Dangguo, Jin, Jianhua, Ma, Lei, Yi, Sanli
• Format: Journal Article
• Language: English
• Published: London Springer London 01.05.2025; Springer Nature B.V
• Subjects: Computer Imaging; Computer Science; Head movement; Heart rate; Image Processing and Computer Vision; Multimedia Information Systems; Noise reduction; Optical flow (image analysis); Original Paper; Pattern Recognition and Graphics; Periodic variations; Physiology; Signal measurement; Signal,Image and Speech Processing; Video; Vision; Remote photoplethysmography; Computer vision; TDCM; Spatio-temporal fusion
• ISSN: 1863-1703; 1863-1711
• DOI: 10.1007/s11760-024-03809-7

Remote physiological measurement estimates heart rate (HR) non-contact by analyzing skin color changes from facial videos. This non-intrusive method can be useful in healthcare, border security, and lie detection. These changes are subtle, imperceptible to the naked eye, and easily affected by variations in lighting and motion artifacts from the subject in front of the camera. Traditional methods often use aggressive noise reduction techniques in the presence of real-world noise, which can obscure heart rate information and lead to distorted detection. This study proposes a more effective method for remote heart rate detection that addresses interference by enhancing facial spatial geometric features using the progressive transformation properties of Recursive Spatial Transformation (ReST). It also integrates motion features obtained from optical flow for more stable spatio-temporal feature enhancement. Additionally, we introduce a Multi-Scale Temporal Convolution Module (TDCM) to capture periodic signal changes across different time scales, modeling the periodicity of heart rate signals from various scales to achieve robust recovery of rPPG signals. The entire model has about 30% of the parameters of PhysFormer. Experiments on multiple remote physiological signal measurement datasets demonstrate that the proposed method significantly improves heart rate estimation across various metrics, particularly showing strong robustness in handling videos with severe head movements.