Bindi: Affective Internet of Things to Combat Gender-Based Violence

• Published in: IEEE internet of things journal Vol. 9; no. 21; pp. 21174 - 21193
• Main Authors: Miranda Calero, Jose A., Rituerto-Gonzalez, Esther, Luis-Mingueza, Clara, Canabal, Manuel F., Barcenas, Alberto Ramirez, Lanza-Gutierrez, Jose M., Pelaez-Moreno, Carmen, Lopez-Ongil, Celia
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.11.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Affective computing; Artificial intelligence; Artificial intelligence of things; Audio data; Cloud computing; Computer architecture; Cyber-physical systems; Data collection; Data integration; Data processing; edge computing; Emotion recognition; Emotions; Fear; fear recognition; Gender-based violence; Internet of Things; microelectromechanical systems; multimodal data fusion; Physiological responses; Physiology; Pipelines; Pipelining (computers); Power consumption; Smart sensors; Training; Violence; Virtual reality; Wearable technology
• ISSN: 2327-4662; 2327-4662
• DOI: 10.1109/JIOT.2022.3177256

The main research motivation of this article is the fight against gender-based violence and achieving gender equality from a technological perspective. The solution proposed in this work goes beyond currently existing panic buttons, needing to be manually operated by the victims under difficult circumstances. Instead, Bindi, our end-to-end autonomous multimodal system, relies on artificial intelligence methods to automatically identify violent situations, based on detecting fear-related emotions, and trigger a protection protocol, if necessary. To this end, Bindi integrates modern state-of-the-art technologies, such as the Internet of Bodies, affective computing, and cyber-physical systems, leveraging: 1) affective Internet of Things (IoT) with auditory and physiological commercial off-the-shelf smart sensors embedded in wearable devices; 2) hierarchical multisensorial information fusion; and 3) the edge-fog-cloud IoT architecture. This solution is evaluated using our own data set named WEMAC, a very recently collected and freely available collection of data comprising the auditory and physiological responses of 47 women to several emotions elicited by using a virtual reality environment. On this basis, this work provides an analysis of multimodal late fusion strategies to combine the physiological and speech data processing pipelines to identify the best intelligence engine strategy for Bindi. In particular, the best data fusion strategy reports an overall fear classification accuracy of 63.61% for a subject-independent approach. Both a power consumption study and an audio data processing pipeline to detect violent acoustic events complement this analysis. This research is intended as an initial multimodal baseline that facilitates further work with real-life elicited fear in women.