Semi-supervised subject recognition in low-modal sensor data

• Published in: Ad hoc networks Vol. 115; p. 102472
• Main Authors: Tiwari, Shivam, Dhekane, Sourish Gunesh, Vajra, Krishnam, Banerjee, Dip Sankar
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 15.04.2021
• Subjects: Pseudo label generation; Semi-supervised learning; Subject Recognition; Transfer learning; Subject Recognition; Semi-supervised learning; Pseudo label generation; Transfer learning
• ISSN: 1570-8705; 1570-8713
• DOI: 10.1016/j.adhoc.2021.102472

Subject Recognition (SR) refers to the task of identifying persons performing activities in a smart environment using the data captured by the sensors installed in it. The existing literature mainly concentrates on supervised SR using the sensor data captured through multiple modalities. However, majority of the real-life sensor datasets are not annotated with the subjects performing the activities, which creates a scarcity of labeled data samples for this task. Issues of privacy and high manual annotation costs further complicate the problem of less labeled data. In addition to this problem, most of the datasets are of low modalities. Hence, the challenge lies in developing semi-supervised frameworks that are suitable for low-modal sensor data with sparse or no labels. Towards this, we initially perform benchmark experiments to analyze the factors of modality and amount of labeled data in the context of SR. Then, we propose semi-supervised frameworks for SR on the data collected by low-modal ubiquitous and visual sensors. In particular, we propose a clustering-based pseudo label generation algorithm to facilitate the training process in a semi-supervised domain for ubiquitous data. On the other hand, we propose Transfer Learning and Data Augmentation (TLDA) framework to perform SR on visual data in semi-supervised domain. To validate our proposed frameworks, we perform experiments on three real-world datasets, namely Smartphone, OPPORTUNITY, and UTD-MHAD dataset to achieve an accuracy of around 77%, 98%, and 91% respectively. Finally, we also provide an analysis on the aspect of merging modalities to propose a new research dimension for SR.