Towards Multimodal VR Trainer of Voice Emission and Public Speaking -Work-in-Progress

• Published in: 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) pp. 355 - 359
• Main Authors: loras-Cvbulska, Maadalena, Hekiert, Daniela, Cybulski, Artur, Tadeia, Slawomir, Witkowski, Marcin, Nakonieczny, Konrad, Augustyn, Izabela, Jasinski, Jan, Hemmerling, Daria, Skrzek, Tomasz, Kacprzak, Stanislaw, Kaczorowska, Magdalena, Juros, Julia, Warzeszka, Marek, Migaczewska, Ewa, Malawski, Filip, Slomka, Paulina, Salamon, Rafal, Szumiec, Aleksandra, Kornacka, Kinga, Blaszczylski, Bartlomiej, Blaszczylska, Katarzyna, Sterna, Radoslaw, Makowiec, Marek, Maidak, Magdalena
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.03.2023
• Subjects: Affective computing-Gestures-Biometry-Eye tracking; Digital signal processing; Three-dimensional displays; Training; User centered design; User interfaces; Virtual reality; Virtual Reality-Speech- Visualization-Voice analysis; Visualization
• DOI: 10.1109/VRW58643.2023.00079

Glossa VR is a virtual reality (VR) application that combines training in public speaking in front of a virtual audience and in voice emission in relaxation exercises. It is accompanied by digital signal processing (DSP) and artificial intelligence (AI) modules which provide automatic feedback on the vocal performance as well as the behavior and psychophysiology of the user. In particular, we address parameters of speech emotions, prosody and timbre, and the user's hand gestures and eye movement. The prototype is in the proof of concept phase, and we are developing it in accordance with the user-centered design paradigm. In this article reports the work in progress, focusing on the approaches, datasets and algorithms applied in the current state of the glossaVR project.