Learning to Reproduce Fluctuating Time Series by Inferring Their Time-Dependent Stochastic Properties: Application in Robot Learning Via Tutoring

• Published in: IEEE transactions on autonomous mental development Vol. 5; no. 4; pp. 298 - 310
• Main Authors: Murata, Shingo, Namikawa, Jun, Arie, Hiroaki, Sugano, Shigeki, Tani, Jun
• Format: Journal Article
• Language: English
• Published: IEEE 01.12.2013
• Subjects: Dynamical systems approach; humanoid robot; Humanoids; Maximum likelihood estimation; Neural networks; recurrent neural network; Recurrent neural networks
• ISSN: 1943-0604; 1943-0612
• DOI: 10.1109/TAMD.2013.2258019

This study proposes a novel type of dynamic neural network model that can learn to extract stochastic or fluctuating structures hidden in time series data. The network learns to predict not only the mean of the next input state, but also its time-dependent variance. The training method is based on maximum likelihood estimation by using the gradient descent method and the likelihood function is expressed as a function of the estimated variance. Regarding the model evaluation, we present numerical experiments in which training data were generated in different ways utilizing Gaussian noise. Our analysis showed that the network can predict the time-dependent variance and the mean and it can also reproduce the target stochastic sequence data by utilizing the estimated variance. Furthermore, it was shown that a humanoid robot using the proposed network can learn to reproduce latent stochastic structures hidden in fluctuating tutoring trajectories. This learning scheme is essential for the acquisition of sensory-guided skilled behavior.