Multimodal System for Diagnosis and Polysensory Stimulation of Subjects with Communication Disorders

• Published in: Foundations of Intelligent Systems Vol. 10352; pp. 47 - 56
• Main Authors: Kurowski, Adam, Odya, Piotr, Szczuko, Piotr, Lech, Michał, Spaleniak, Paweł, Kostek, Bożena, Czyżewski, Andrzej
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2017; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Communication disorders; Data clustering; EEG; Electroencephalography; Multimodal interfaces; Polysensory stimulation; Signal segmentation; Special education
• ISBN: 9783319604374; 3319604376
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-319-60438-1_5

An experimental multimodal system, designed for polysensory diagnosis and stimulation of persons with impaired communication skills or even non-communicative subjects is presented. The user interface includes an eye tracking device and the EEG monitoring of the subject. Furthermore, the system consists of a device for objective hearing testing and an autostereoscopic projection system designed to stimulate subjects through their immersion in a virtual environment. Data analysis methods are described, and experiments associated with classification of mental states during listening exercises as well as audio-visual stimuli are presented and discussed. Feature extraction was based on discrete wavelet transformation and clustering employing the k-means algorithm was designed. All algorithms were implemented in the Python programming language with the use of Open Source libraries. Tests of the proposed system were performed in a Special School and Educational Center in Kościerzyna, Poland. Results and comparison with data gathered from the control group of healthy people are presented and discussed.