Federated learning inspired privacy sensitive emotion recognition based on multi-modal physiological sensors

Traditional machine learning classifiers can automatically evaluate human behaviour and emotion recognition tasks. However, prior research work does not secure users’ privacy and personal information because they need complete access to sensitive physiological data. The recently introduced Federated...

Full description

Saved in:
Bibliographic Details
Published inCluster computing Vol. 27; no. 3; pp. 3179 - 3201
Main Authors Gahlan, Neha, Sethia, Divyashikha
Format Journal Article
LanguageEnglish
Published New York Springer US 01.06.2024
Subjects
Computer Communication Networks
Computer Science
Operating Systems
Processor Architectures
MLP
Physiological sensors
Privacy
Federated learning
Multi-modal
Emotion recognition system
Online AccessGet full text
ISSN1386-7857
1573-7543
DOI10.1007/s10586-023-04133-4

Cover

More Information
Summary:Traditional machine learning classifiers can automatically evaluate human behaviour and emotion recognition tasks. However, prior research work does not secure users’ privacy and personal information because they need complete access to sensitive physiological data. The recently introduced Federated Learning (FL) paradigm can address this problem. FL allows the local model updates to be sent to a central server, combining them to create a global model. It does not allow the global model to access the raw data used to train it. Motivated by the core concept of FL, this paper proposes a novel FL-based Multi-modal Emotion Recognition System (F-MERS) framework combining EEG, GSR, ECG, and RESP physiological sensors data. It uses Multi-layer Perceptron (MLP) as a base model for classifying complex emotions in three dimensions: Valence, Arousal, and Dominance (VAD). The work validates the F-MERS framework with three emotion benchmark datasets, DEAP, AMIGOS, and DREAMER, achieving accuracies of 87.90%, 89.02%, and 79.02%, respectively. It is the first FL-enabled framework for recognizing complex emotions in three dimensions (VAD) with multi-modal physiological sensors. The proposed study assesses the F-MERS framework in two scenarios: (1). Subject dependent and (2). Subject independent, making the framework more generalized and robust. The experimental outcomes indicate that the F-MERS framework is scalable, efficient in communication, and offers privacy preservation over the baseline Non-FL MLP model.
ISSN:1386-7857
1573-7543
DOI:10.1007/s10586-023-04133-4