FamilyLog: A mobile system for monitoring family mealtime activities

• Published in: Proceedings of the ... IEEE International Conference on Pervasive Computing and Communications. IEEE International Conference on Pervasive Computing and Communications Vol. 2017; pp. 21 - 30
• Main Authors: Chongguang Bi, Xing, Guoliang, Hao, Tian, Jina Huh, Wei Peng, Mengyan Ma
• Format: Conference Proceeding Journal Article
• Language: English
• Published: United States IEEE 01.03.2017
• Subjects: Acoustics; Feature extraction; Monitoring; Smart phones
• ISSN: 2474-2503; 2474-249X
• DOI: 10.1109/PERCOM.2017.7917847

Research has shown that family mealtime plays a critical role in establishing good relationships among family members and maintaining their physical and mental health. In particular, regularly eating dinner as a family significantly reduces prevalence of obesity. However, American families with children spend only 1 hour on family meals while three hours watching TV on an average work day. Fine-grained activity-logging is proven effective for increasing self-awareness and motivating people to modify their life styles for improved wellness. This paper presents FamilyLog - a practical system to log family mealtime activities using smartphones and smartwatches. FamilyLog automatically detects and logs details of activities during the mealtime, including occurrence and duration of meal, conversations, participants, TV viewing etc., in an unobtrusive manner. Based on the sensor data collected from real families, we carefully design robust yet lightweight signal features from a set of complex activities during the meal, including clattering sound, arm gestures of eating, human voice, TV sound, etc. Moreover, FamilyLog opportunistically fuses data from built-in sensors of multiple mobile devices available in a family through an HMM-based classifier. To evaluate the real-world performance of FamilyLog, we perform extensive experiments that consist of 77 days of sensor data from 37 subjects in 8 families with children. Our results show that FamilyLog can detect those events with high accuracy across different families and home environments.