Intelligent affect regression for bodily expressions using hybrid particle swarm optimization and adaptive ensembles

• Published in: Expert systems with applications Vol. 42; no. 22; pp. 8678 - 8697
• Main Authors: Zhang, Yang, Zhang, Li, Neoh, Siew Chin, Mistry, Kamlesh, Hossain, Mohammed Alamgir
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.12.2015
• Subjects: Adaptive ensemble regression; Algorithms; Arousal; Bodily expression; Genetic algorithm; Genetic algorithms; Mutation distributions; Mutations; Optimization; Particle swarm optimization; Regression; State of the art; Support vector regression; Swarm intelligence; Support vector regression; Adaptive ensemble regression; Genetic algorithm; Mutation distributions; Particle swarm optimization; Bodily expression
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2015.07.022

•We conduct dimensional affect recognition of bodily expressions.•Hybrid particle swarm optimization (PSO) is proposed for feature selection.•It mitigates premature convergence problem of conventional PSO.•Mutation mechanisms of each subswarm work cooperatively to avoid stagnation.•Our system outperforms other PSO-based and bodily expression perception research. This research focuses on continuous dimensional affect recognition from bodily expressions using feature optimization and adaptive regression. Both static posture and dynamic motion bodily features are extracted in this research. A hybrid particle swarm optimization (PSO) algorithm is proposed for feature selection, which overcomes premature convergence and local optimum trap encountered by conventional PSO. It integrates diverse jump-out mechanisms such as the genetic algorithm (GA) and mutation techniques of Gaussian, Cauchy and Levy distributions to balance well between convergence speed and swarm diversity, thus called GM-PSO. The proposed PSO variant employs the subswarm concept and a cooperative strategy to enable mutation mechanisms of each subswarm, i.e. the GA and the probability distributions, to work in a collaborative manner to enhance the exploration and exploitation capability of the swarm leader, sustain the population diversity and guide the search toward an ultimate global optimum. An adaptive ensemble regression model is subsequently proposed to robustly map subjects’ emotional states onto a continuous arousal–valence affective space using the identified optimized feature subsets. This regression model also shows great adaption to newly arrived bodily expression patterns to deal with data stream regression. Empirical findings indicate that the proposed hybrid PSO optimization algorithm outperforms other state-of-the-art PSO variants, conventional PSO and classic GA significantly in terms of catching global optimum and discriminative feature selection. The system achieves the best performance for the regression of arousal and valence when ensemble regression model is applied, in terms of both mean squared error (arousal: 0.054, valence: 0.08) and Pearson correlation coefficient (arousal: 0.97, valence: 0.91) and outperforms other state-of-the-art PSO-based optimization combined with ensemble regression and related bodily expression perception research by a significant margin.