An AI-Edge Platform with Multimodal Wearable Physiological Signals Monitoring Sensors for Affective Computing Applications

• Published in: 2020 IEEE International Symposium on Circuits and Systems (ISCAS) pp. 1 - 5
• Main Authors: Yang, Cheng-Jie, Fahier, Nicolas, He, Chang-Yuan, Li, Wei-Chih, Fang, Wai-Chi
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.10.2020
• Subjects: affective computing; Biomedical monitoring; Brain modeling; convolutional neural network; Electrocardiography; Electroencephalography; Emotion recognition; Feature extraction; Monitoring; multimodal analysis; physiological signals
• ISBN: 9781728133201; 1728133203
• ISSN: 2158-1525; 2158-1525
• DOI: 10.1109/ISCAS45731.2020.9180909

In this paper, we developed and integrated an AI-edge emotion recognition platform using multiple wearable physiological signals sensors: Electroencephalogram (EEG), electrocardiogram (ECG), and photoplethysmogram (PPG) sensors. The emotion recognition platform used two combined machine learning approaches based on two systems input and preprocessing: An EEG-based emotion recognition system and an ECG/PPG-based system. The EEG-based system is a convolution neural network (CNN) that classifies three emotions, happiness, anger and sadness. The inputs of the CNN are extracted from the EEG signals using short-time Fourier transform (STFT), and the average accuracy for a subject-independent classification reached 76.94%. The ECG/PPG-based system used a similar CNN with an extracted features vector as input. The subject-dependent ECG/PPG classification system reached an average accuracy of 76.8%. The proposed system was integrated using the RISC-V processor and FPGA platforms to implement realtime monitoring and classification on edge. A 3-to-1 Bluetooth piconet was deployed to transmit all physiological signals on a single platform access point and to make use of low power wireless technologies.