Learning linear parameter-varying control of small-scale helicopter using episodic natural actor-critic method

• Published in: 2012 12th International Conference on Control Automation Robotics & Vision (ICARCV) pp. 1001 - 1005
• Main Authors: Zhou Fang, Chuanchuan Hao, Ping Li
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.12.2012
• Subjects: Convergence; flight control; helicopter; Helicopters; Learning (artificial intelligence); LPV control; Natural actor-critic; Nonlinear dynamical systems; reinforcement learning; Robots; Scheduling; State feedback
• ISBN: 9781467318716; 146731871X
• DOI: 10.1109/ICARCV.2012.6485294

In this paper, we present the application of episodic natural actor-critic (eNAC) method to learn an LPV controller for the longitudinal motion of a small-scale helicopter. Rewards are online constructed using the quadratic error between the real output and desired output from reference model. The control law is given in a gain-scheduled PID form to accommodate nonlinear longitudinal dynamics. Natural gradients are computed while interacting with the environment, and scaling technique is applied to the gradient components associated with the scheduling parameters to improve the convergence rate of the learning process. To handle the partial observability of the small helicopter's dynamics, notch filter is connected in tandem to the longitudinal cyclic in advance. When performing cruise flight in nonlinear simulation, the learned LPV controller shows better performance in the respect of reference tracking, compared to the conventional PID controller, demonstrating the effectiveness of our method.