Consciousness-Inspired Spatio-Temporal Abstractions for Better Generalization in Reinforcement Learning

Inspired by human conscious planning, we propose Skipper, a model-based reinforcement learning framework utilizing spatio-temporal abstractions to generalize better in novel situations. It automatically decomposes the given task into smaller, more manageable subtasks, and thus enables sparse decisio...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Zhao, Mingde, Alver, Safa, Harm van Seijen, Laroche, Romain, Precup, Doina, Bengio, Yoshua
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 16.03.2024
Subjects
Apexes
Graph theory
Graphical representations
Online AccessGet full text

Cover

More Information
Summary:Inspired by human conscious planning, we propose Skipper, a model-based reinforcement learning framework utilizing spatio-temporal abstractions to generalize better in novel situations. It automatically decomposes the given task into smaller, more manageable subtasks, and thus enables sparse decision-making and focused computation on the relevant parts of the environment. The decomposition relies on the extraction of an abstracted proxy problem represented as a directed graph, in which vertices and edges are learned end-to-end from hindsight. Our theoretical analyses provide performance guarantees under appropriate assumptions and establish where our approach is expected to be helpful. Generalization-focused experiments validate Skipper's significant advantage in zero-shot generalization, compared to some existing state-of-the-art hierarchical planning methods.
ISSN:2331-8422