Neuro-SERKET: Development of Integrative Cognitive System Through the Composition of Deep Probabilistic Generative Models

• Published in: New generation computing Vol. 38; no. 1; pp. 23 - 48
• Main Authors: Taniguchi, Tadahiro, Nakamura, Tomoaki, Suzuki, Masahiro, Kuniyasu, Ryo, Hayashi, Kaede, Taniguchi, Akira, Horii, Takato, Nagai, Takayuki
• Format: Journal Article
• Language: English
• Published: Tokyo Ohmsha 01.03.2020; Springer Nature B.V
• Subjects: Artificial Intelligence; Automatic speech recognition; Computer Hardware; Computer Science; Computer Systems Organization and Communication Networks; Dirichlet problem; Neural networks; Probabilistic models; Software Engineering/Programming and Operating Systems; Symbol emergence in robotics; Cognitive models; Deep generative models; Machine learning; Probabilistic generative models
• ISSN: 0288-3635; 1882-7055
• DOI: 10.1007/s00354-019-00084-w

This paper describes a framework for the development of an integrative cognitive system based on probabilistic generative models (PGMs) called Neuro-SERKET. Neuro-SERKET is an extension of SERKET, which can compose elemental PGMs developed in a distributed manner and provide a scheme that allows the composed PGMs to learn throughout the system in an unsupervised way. In addition to the head-to-tail connection supported by SERKET, Neuro-SERKET supports tail-to-tail and head-to-head connections, as well as neural network-based modules, i.e., deep generative models. As an example of a Neuro-SERKET application, an integrative model was developed by composing a variational autoencoder (VAE), a Gaussian mixture model (GMM), latent Dirichlet allocation (LDA), and automatic speech recognition (ASR). The model is called VAE + GMM + LDA + ASR. The performance of VAE + GMM + LDA + ASR and the validity of Neuro-SERKET were demonstrated through a multimodal categorization task using image data and a speech signal of numerical digits.