Self-feeding frequency estimation and eating action recognition from skeletal representation using Kinect

• Published in: World wide web (Bussum) Vol. 22; no. 3; pp. 1343 - 1358
• Main Authors: Men, Qianhui, Leung, Howard, Yang, Yang
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.05.2019; Springer Nature B.V
• Subjects: Acceleration; Activity recognition; Computer Science; Database Management; Drinking; Eating; Eating behavior; Estimation; Feature extraction; Information Systems Applications (incl.Internet); Operating Systems; Sensors; Skeletal representation; Special Issue on Social Media and Interactive Technologies; Eating action recognition; Microsoft Kinect; Skeleton motions; Self-feeding detection
• ISSN: 1386-145X; 1573-1413
• DOI: 10.1007/s11280-018-0567-0

Under healthcare research, eating activity detection and recognition have been studied for many years. Most of the previous approaches rely on body worn sensors for eating behavior detection. However, measurement errors from these sensors will largely reduce the tracking accuracy in estimating velocity and acceleration. To avoid this problem, we utilize Microsoft Kinect to capture skeleton motions of eating and drinking behaviors. In this paper we introduce a moving average method to remove the noise in relative distances of the captured joint positions so that eating activity can be segmented into feeding and non-feeding frames. In order to identify different eating patterns, eating and drinking behavior recognition is performed based on the features extracted from the resulting feeding periods. The experiments are evaluated on our collected eating and drinking action dataset, and we also change the distance between Kinect and subject to test the robustness of our approach. The results achieve better detection and recognition performance compared with other approaches. This pioneer work of our eating action behavior analysis can lead to many potential applications such as the development of a Web system to facilitate people to share and search their eating and drinking actions as well as carrying out intelligent analysis to provide suggestions.