A deep joint sparse non-negative matrix factorization framework for identifying the common and subject-specific functional units of tongue motion during speech

• Published in: Medical image analysis Vol. 72; p. 102131
• Main Authors: Woo, Jonghye, Xing, Fangxu, Prince, Jerry L., Stone, Maureen, Gomez, Arnold D., Reese, Timothy G., Wedeen, Van J., El Fakhri, Georges
• Format: Journal Article
• Language: English
• Published: Netherlands Elsevier B.V 01.08.2021; Elsevier BV
• Subjects: Algorithms; Artificial neural networks; Clustering; Coordination; Coordination compounds; Deep learning; Deep non-negative matrix factorization; Factorization; Functional units; Humans; In vivo methods and tests; Joints (anatomy); Machine learning; Magnetic Resonance Imaging; Muscles; Neural networks; Neural Networks, Computer; Speech; Structure-function relationships; Tagged-MRI; Tongue; Tongue - diagnostic imaging; Tongue motion; Weighting; Tagged-MRI; Functional units; Deep non-negative matrix factorization; Tongue motion
• ISSN: 1361-8415; 1361-8423; 1361-8423
• DOI: 10.1016/j.media.2021.102131

•We presented a method to identify both common and subject-specific functional units within a material coordinate system from MRI.•We proposed to convert non-negative matrix factorization with sparse and manifold regularizations into modular structures.•We provided validation to demonstrate superior performance over the comparison methods on both simulated and in vivo tongue data. [Display omitted] Intelligible speech is produced by creating varying internal local muscle groupings—i.e., functional units—that are generated in a systematic and coordinated manner. There are two major challenges in characterizing and analyzing functional units. First, due to the complex and convoluted nature of tongue structure and function, it is of great importance to develop a method that can accurately decode complex muscle coordination patterns during speech. Second, it is challenging to keep identified functional units across subjects comparable due to their substantial variability. In this work, to address these challenges, we develop a new deep learning framework to identify common and subject-specific functional units of tongue motion during speech. Our framework hinges on joint deep graph-regularized sparse non-negative matrix factorization (NMF) using motion quantities derived from displacements by tagged Magnetic Resonance Imaging. More specifically, we transform NMF with sparse and graph regularizations into modular architectures akin to deep neural networks by means of unfolding the Iterative Shrinkage-Thresholding Algorithm to learn interpretable building blocks and associated weighting map. We then apply spectral clustering to common and subject-specific weighting maps from which we jointly determine the common and subject-specific functional units. Experiments carried out with simulated datasets show that the proposed method achieved on par or better clustering performance over the comparison methods.Experiments carried out with in vivo tongue motion data show that the proposed method can determine the common and subject-specific functional units with increased interpretability and decreased size variability.