Fusing Partial Camera Signals for Noncontact Pulse Rate Variability Measurement

• Published in: IEEE transactions on biomedical engineering Vol. 65; no. 8; pp. 1725 - 1739
• Main Authors: McDuff, Daniel J., Blackford, Ethan B., Estepp, Justin R.
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.08.2018; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adolescent; Adult; Affective computing; Biomedical monitoring; camera; Cameras; Computer vision; Data loss; Data processing; Errors; Face; Face - diagnostic imaging; Female; Head movement; Humans; Image color analysis; Image Processing, Computer-Assisted - methods; Male; Median (statistics); non-contact; Parameter estimation; Photoplethysmography (PPG); Photoplethysmography - methods; Physiology; Pulse - methods; Pulse measurements; Pulse rate; pulse rate variability (PRV); Signal processing; Signal Processing, Computer-Assisted; Test procedures; Windows (intervals); Young Adult
• DOI: 10.1109/TBME.2017.2771518
• ISSN: 0018-9294; 1558-2531

Remote camera-based measurement of physiology has great potential for healthcare and affective computing. Recent advances in computer vision and signal processing have enabled photoplethysmography (PPG) measurement using commercially available cameras. However, there remain challenges in recovering accurate noncontact PPG measurements in the presence of rigid head motion. When a subject is moving, their face may be turned away from one camera, be obscured by an object, or move out of the frame resulting in missing observations. As the calculation of pulse rate variability (PRV) requires analysis over a time window of several minutes, the effect of missing observations on such features is deleterious. We present an approach for fusing partial color-channel signals from an array of cameras that enable physiology measurements to be made from moving subjects, even if they leave the frame of one or more cameras, which would not otherwise be possible with only a single camera. We systematically test our method on subjects (N=25) using a set of six, 5-min tasks (each repeated twice) involving different levels of head motion. This results in validation across 25 h of measurement. We evaluate pulse rate and PRV parameter estimation including statistical, geometric, and frequency-based measures. The median absolute error in pulse rate measurements was 0.57 beats-per-minute (BPM). In all but two tasks with the greatest motion, the median error was within 0.4 BPM of that from a contact PPG device. PRV estimates were significantly improved using our proposed approach compared to an alternative not designed to handle missing values and multiple camera signals; the error was reduced by over 50%. Without our proposed method, errors in pulse rate would be very high, and estimation of PRV parameters would not be feasible due to significant data loss.