A Reinforcement Learning Method for Quadrotor Attitude Control Based on Expert Information

In this paper, a model-free reinforcement learning(RL) method of training a nonlinear attitude controller of a quadrotor is proposed. For the problem that the attitude controller is uncontrolled when trained by RL directly, the proposed method utilizes an expert to provide the prior information, i.e...

Full description

Saved in:
Bibliographic Details
Published in2023 8th International Conference on Automation, Control and Robotics Engineering (CACRE) pp. 281 - 286
Main Authors Zhu, Yalu, Lian, Shikang, Zhong, WenTao, Meng, Wei
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.07.2023
Subjects
Approximation algorithms
Attitude control
Entropy
Expert information
Heuristic algorithms
PID
Process control
Proximal policy optimization
Quadrotor
Reinforcement learning
Switches
Training
Online AccessGet full text

Cover

More Information
Summary:In this paper, a model-free reinforcement learning(RL) method of training a nonlinear attitude controller of a quadrotor is proposed. For the problem that the attitude controller is uncontrolled when trained by RL directly, the proposed method utilizes an expert to provide the prior information, i.e. the action's judgement and suggestion, to guide the updating process. For the problem that the policy falls in local optima by the limitation of the expert, the proposed method maximize the entropy of the strategy to increase the exploratory behavior of the nonlinear attitude controller approximator. Furthermore, We employ the Proximal policy optimization algorithm (PPO) as the RL model and PID algorithm as the expert model to approach an exact attitude controller of a quadrotor based on the proposed method. Finally, the simulations experiments has been conducted to verify that our proposed method can train a true nonlinear attitude controller which has a better performance than the expert.
DOI:10.1109/CACRE58689.2023.10208497