A neonatal dataset and benchmark for non-contact neonatal heart rate monitoring based on spatio-temporal neural networks

• Published in: Engineering applications of artificial intelligence Vol. 106; p. 104447
• Main Authors: Huang, Bin, Chen, Weihai, Lin, Chun-Liang, Juang, Chia-Feng, Xing, Yuanping, Wang, Yanting, Wang, Jianhua
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.11.2021
• Subjects: Deep learning; Neonatal health care; Non-contact heart rate monitoring; Physiological signal monitoring; Remote photoplethysmograph (rPPG); Deep learning; Non-contact heart rate monitoring; Physiological signal monitoring; Neonatal health care; Remote photoplethysmograph (rPPG)
• ISSN: 0952-1976; 1873-6769
• DOI: 10.1016/j.engappai.2021.104447

The digital revolution of noncontact physiological signal monitoring in clinical and home health care is underway, and deep learning techniques are incredibly popular. Camera-based physiological signal monitoring for adults has made considerable progress in recent years. However, most of existing methods and datasets are developed for adult subjects, and until now, there has been no neonatal public database that is collected for developing deep learning method. Thus, in this paper, we introduce a large-scale newborn baby database, named NBHR (newborn baby heart rate estimation database), to fill the abovementioned knowledge gap. A total of 9.6 h of clinical videos (1130 videos totaling 921 GB) and reference vital signs are recorded from 257 infants at 0–6 days old. The facial videos and corresponding synchronized physiological signals, including photoplethysmograph information, heart rate, and oxygen saturation level, are recorded in our database. This large-scale database could be used to develop deep learning methods to estimate heart rate or oxygen saturation levels. Furthermore, a multitask deep learning method, called NBHRnet, is proposed to estimate heart rate based on the NBHR database, and the model is succinct that it can be deployed on a computer without GPUs. The experimental results indicate that NBHRnet yields competitive performance in predicting infant heart rate, with a mean absolute error of 3.97 bpm and a mean absolute percentage error of 3.28%; additionally, it can estimate heart rate almost instantaneously (2 s/60 frames). Our datasets are freely publicly available by request. •NBHR is the first open large-scale dataset that recorded newborn babies’ physiological signal.•Compared with adult’ dataset, NBHR is recorded in clinical and real-world environment.•A novel spatio-temporal neural network is proposed for extracting rPPG and HR.•A novel loss function and training strategy are developed for optimizing NBHRnet.•The study fills the research gap of non-contact neonatal monitoring based on deep learning.