Embedding a priori knowledge in reinforcement learning

• Published in: Journal of intelligent & robotic systems Vol. 21; no. 1; pp. 51 - 71
• Main Author: RIBEIRO, C. H. C
• Format: Journal Article
• Language: English
• Published: Dordrecht Kluwer 1998; Springer Nature B.V
• Subjects: Adaptive algorithms; Algorithms; Applied sciences; Artificial intelligence; Computer science; control theory; systems; Convergence; Exact sciences and technology; Exploration; Learning; Learning and adaptive systems; Mathematical analysis; Reagents; Reinforcement; Studies; Temporal logic; Learning; Algorithm; Reinforcement learning
• ISSN: 0921-0296; 1573-0409
• DOI: 10.1023/A:1007968115863

In the last years, temporal differences methods have been put forward as convenient tools for reinforcement learning. Techniques based on temporal differences, however, suffer from a serious drawback: as stochastic adaptive algorithms, they may need extensive exploration of the state-action space before convergence is achieved. Although the basic methods are now reasonably well understood, it is precisely the structural simplicity of the reinforcement learning principle - learning through experimentation - that causes these excessive demands on the learning agent. Additionally, one must consider that the agent is very rarely a tabula rasa: some rough knowledge about characteristics of the surrounding environment is often available. In this paper, I present methods for embedding a priori knowledge in a reinforcement learning technique in such a way that both the mathematical structure of the basic learning algorithm and the capacity to generalise experience across the state-action space are kept. Extensive experimental results show that the resulting variants may lead to good performance, provided a sensible balance between risky use of prior imprecise knowledge and cautious use of learning experience is adopted.[PUBLICATION ABSTRACT]