Online learning control by association and reinforcement

• Published in: IEEE transactions on neural networks Vol. 12; no. 2; pp. 264 - 276
• Main Authors: Si, J., Yu-Tsung Wang
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.03.2001
• Subjects: Control design; Control systems; Dynamic programming; Dynamical systems; Dynamics; Guidelines; Learning; Learning systems; On-line systems; Online; Performance enhancement; Real time systems; Reinforcement; Stochastic systems; System performance; System testing; Velocity measurement
• ISSN: 1045-9227; 1941-0093
• DOI: 10.1109/72.914523

This paper focuses on a systematic treatment for developing a generic online learning control system based on the fundamental principle of reinforcement learning or more specifically neural dynamic programming. This online learning system improves its performance over time in two aspects: 1) it learns from its own mistakes through the reinforcement signal from the external environment and tries to reinforce its action to improve future performance; and 2) system states associated with the positive reinforcement is memorized through a network learning process where in the future, similar states will be more positively associated with a control action leading to a positive reinforcement. A successful candidate of online learning control design is introduced. Real-time learning algorithms is derived for individual components in the learning system. Some analytical insight is provided to give guidelines on the learning process took place in each module of the online learning control system.