Large-scale Investigations of AAC Usage Patterns: Trends, Autism, and Stacked Autoencoders

• Published in: 2023 IEEE 13th Annual Computing and Communication Workshop and Conference (CCWC) pp. 851 - 859
• Main Authors: Atyabi, Adham, Boccanfuso, Laura, Snider, J.C., Kim, Minah, Barney, Erin, Ahn, Yeojin Amy, Li, Beibin, Dommer, Kelsey Jackson, Shic, Frederick
• Format: Conference Proceeding
• Language: English
• Published: IEEE 08.03.2023
• Subjects: Autism; Data models; Numerical models; Sociology; Stability analysis; Statistics
• DOI: 10.1109/CCWC57344.2023.10099096

Augmentative and Alternative Communication (AAC) applications are platforms that provide means for non-speech communication. One class of AAC apps are speech-generating devices (SGDs). These AAC apps utilize icons/pictures that when tapped, spoken words will be produced. These apps are widely used to support communication and language learning for individuals with disabilities such as autism spectrum disorder (ASD). While the common mechanism for data collection in individuals with disabilities such as ASD is complex, requires specific technical mastery in acquisition site's staff, and often cumbersome for participants, the daily use of AAC apps (in home settings and at will) provides opportunity to collect large scales of usage patterns that contains a wealth of information regarding individuals' usage patterns that are essential components in developing usage model profiles. Despite such potential, the utility of these streams of data are rarely explored from a data science and data modeling perspective. This study provides a comprehensive view on various mechanisms and methodologies for modeling the usage patterns and their potential for identifying differences in AAC usage patterns between users with and without ASD. Data streams acquired from an AAC app called FreeSpeech is used in the study. FreeSpeech is an AAC app that is specifically designed for individuals with learning disabilities and ASD. Data modeling mechanism is initiated using a pseudo-semantic representation of touched words and other events via unique numeric values. Both supervised and unsupervised learning methods are considered in the study. Temporal, sequential, and spectral usage pattern modelings are considered among which modeling mechanisms based on sequential key-press representations outperformed others achieving 70% diagnosis classification accuracy using support-vector-machine. This performance is improved to 82% accuracy using deep learning methods such as stacked auto-encoders.