Deep-Learning Technique for Risk-based Action Prediction Using Extremely Low-Resolution Thermopile Sensor Array

• Published in: IEEE transactions on circuits and systems for video technology Vol. 33; no. 6; p. 1
• Main Authors: Morawski, Igor, Lie, Wen-Nung, Aing, Lee, Chiang, Jui-Chiu, Chen, Kuan-Ting
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.06.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accidents; action prediction; Artificial neural networks; Cameras; Deep learning; elderly monitoring; low resolution; Machine learning; Monitoring; Neural networks; Older adults; Older people; privacy; Real time operation; Recurrent neural networks; Sensor arrays; Sensors; Sociology; Statistics; Task analysis; thermopile sensor array; Thermopiles
• DOI: 10.1109/TCSVT.2022.3229059
• ISSN: 1051-8215

Eldering caring is important in today's aging society, especially that accident anticipation/prevention plays an important role. In this paper, a novel approach to preventing elderly accidents based on a very low-resolution thermopile sensor array (TPA) (only 32×32 pixels) is proposed for prediction of bed-exit event that might lead to elderly falls in home caring. Low-resolution TPA sensor, capable of collecting far infrared energy, ensures cost-effective monitoring, no interference with user's daily life, and most importantly privacy-preservation. Since most of the fall accidents occur when the elderly attempts to get off the bed without assistance, it is thus the focus of this paper to monitor his/her posture and action via TPA image sensor and then predict that an action of getting off the bed will occur in a near future (e.g., S seconds later). Our system can raise an alarm to the caregivers so that they can intervene and offer the necessary assistance. A deep-learning model based on CNN-RNN (Convolutional neural network-Recurrent neural network) architecture was designed which is capable of predicting the elderly bed-exit intention by S = 5.78 seconds in advance of the action onset at an accuracy of 99.37% according to our dataset evaluation. Our system is also suitable for on-line real-time operation which will be helpful to elderly caring in our society.