Adaptive dynamic programming for linear impulse systems

We investigate the optimization of linear impulse systems with the reinforcement learning based adaptive dynamic programming （ADP） method. For linear impulse systems, the optimal objective function is shown to be a quadric form of the pre-impulse states. The ADP method provides solutions that iterat...

Full description

Saved in:
Bibliographic Details
Published inFrontiers of information technology & electronic engineering Vol. 15; no. 1; pp. 43 - 50
Main Authors Wang, Xiao-hua, Yu, Juan-juan, Huang, Yao, Wang, Hua, Miao, Zhong-hua
Format Journal Article
LanguageEnglish
Published Berlin/Heidelberg Springer Berlin Heidelberg 2014
Springer Nature B.V
Subjects
Communications Engineering
Computer Hardware
Computer Science
Computer Systems Organization and Communication Networks
Convergence
Dynamic programming
Electrical Engineering
Electronics and Microelectronics
Functions (mathematics)
Instrumentation
Networks
Neural networks
Optimal control
Polynomials
Quadratic forms
Impulse system
Neural network
TP273.1
Adaptive dynamic programming (ADP)
Optimal control
Online AccessGet full text

Cover

More Information
Summary:We investigate the optimization of linear impulse systems with the reinforcement learning based adaptive dynamic programming （ADP） method. For linear impulse systems, the optimal objective function is shown to be a quadric form of the pre-impulse states. The ADP method provides solutions that iteratively converge to the optimal objective function. If an initial guess of the pre-impulse objective function is selected as a quadratic form of the pre-impulse states, the objective function iteratively converges to the optimal one through ADP. Though direct use of the quadratic objective function of the states within the ADP method is theoretically possible, the numerical singularity problem may occur due to the matrix inversion therein when the system dimensionality increases. A neural network based ADP method can circumvent this problem. A neural network with polynomial activation functions is selected to approximate the pr~impulse objective function and trained iteratively using the ADP method to achieve optimal control. After a successful training, optimal impulse control can be derived. Simulations are presented for illustrative purposes.
Bibliography:We investigate the optimization of linear impulse systems with the reinforcement learning based adaptive dynamic programming （ADP） method. For linear impulse systems, the optimal objective function is shown to be a quadric form of the pre-impulse states. The ADP method provides solutions that iteratively converge to the optimal objective function. If an initial guess of the pre-impulse objective function is selected as a quadratic form of the pre-impulse states, the objective function iteratively converges to the optimal one through ADP. Though direct use of the quadratic objective function of the states within the ADP method is theoretically possible, the numerical singularity problem may occur due to the matrix inversion therein when the system dimensionality increases. A neural network based ADP method can circumvent this problem. A neural network with polynomial activation functions is selected to approximate the pr~impulse objective function and trained iteratively using the ADP method to achieve optimal control. After a successful training, optimal impulse control can be derived. Simulations are presented for illustrative purposes.
Adaptive dynamic programming （ADP）, Impulse system, Optimal control, Neural network
ISSN:1869-1951
2095-9184
1869-196X
2095-9230
DOI:10.1631/jzus.C1300145