Cross-environment activity recognition using word embeddings for sensor and activity representation

• Published in: Neurocomputing (Amsterdam) Vol. 418; pp. 280 - 290
• Main Authors: Azkune, Gorka, Almeida, Aitor, Agirre, Eneko
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 22.12.2020
• Subjects: Activity recognition; Cross-environment activity recognition; Semantic representations; Smart homes; Activity recognition; Cross-environment activity recognition; Smart homes; Semantic representations
• ISSN: 0925-2312; 1872-8286
• DOI: 10.1016/j.neucom.2020.08.044

•We present a novel representation approach for sensors and activities in smart homes which captures the semantic information of those using neural word embeddings.•We develop two machine learning systems – an unsupervised and a supervised one – on top of the proposed representation approach which can tackle the problem of cross-environment activity recognition.•We test both systems in four well-known smart home datasets outperforming three baselines.•We show the potential of our representation approach combined with learning to tackle the cross-environment activity recognition problem. Cross-environment activity recognition in smart homes is a very challenging problem, specially for data-driven approaches. Currently, systems developed to work for a certain environment degrade substantially when applied to a new environment, where not only sensors, but also the monitored activities may be different. Some systems require manual labeling and mapping of the new sensor names and activities using an ontology. Ideally, given a new smart home, we would like to be able to deploy the system, which has been trained on other sources, with minimal manual effort and with acceptable performance. In this paper, we propose the use of neural word embeddings to represent sensor activations and activities, which comes with several advantages: (i) the representation of the semantic information of sensor and activity names, and (ii) automatically mapping sensors and activities of different environments into the same semantic space. Based on this novel representation approach, we propose two data-driven activity recognition systems: the first one is a completely unsupervised system based on embedding similarities, while the second one adds a supervised learning regressor on top of them. We compare our approaches with some baselines using four public datasets, showing that data-driven cross-environment activity recognition obtains good results even when sensors and activity labels significantly differ. Our results show promise for reducing manual effort, and are complementary to other efforts using ontologies.