Federated learning inspired privacy sensitive emotion recognition based on multi-modal physiological sensors - Gahlan, Neha; Sethia, Divyashikha (Cluster computing, 2024)
Traditional machine learning classifiers can automatically evaluate human behaviour and emotion recognition tasks. However, prior research work does not secure users’ privacy and personal information because they need complete access to sensitive physiological data. The recently introduced Federated Learning (FL) paradigm can address this problem. FL allows the local model updates to be sent to a central server, combining them to create a global model. It does not allow the global model to access the raw data used to train it. Motivated by the core concept of FL, this paper proposes a novel FL-based Multi-modal Emotion Recognition System (F-MERS) framework combining EEG, GSR, ECG, and RESP physiological sensors data. It uses Multi-layer Perceptron (MLP) as a base model for classifying complex emotions in three dimensions: Valence, Arousal, and Dominance (VAD). The work validates the F-MERS framework with three emotion benchmark datasets, DEAP, AMIGOS, and DREAMER, achieving accuracies of 87.90%, 89.02%, and 79.02%, respectively. It is the first FL-enabled framework for recognizing complex emotions in three dimensions (VAD) with multi-modal physiological sensors. The proposed study assesses the F-MERS framework in two scenarios: (1). Subject dependent and (2). Subject independent, making the framework more generalized and robust. The experimental outcomes indicate that the F-MERS framework is scalable, efficient in communication, and offers privacy preservation over the baseline Non-FL MLP model.