BYOL-Explore: Exploration by Bootstrapped Prediction

• Main Authors: Guo, Zhaohan Daniel, Thakoor, Shantanu, Pîslar, Miruna, Pires, Bernardo Avila, Altché, Florent, Tallec, Corentin, Saade, Alaa, Calandriello, Daniele, Grill, Jean-Bastien, Tang, Yunhao, Valko, Michal, Munos, Rémi, Azar, Mohammad Gheshlaghi, Piot, Bilal
• Format: Journal Article
• Language: English
• Published: 16.06.2022
• Subjects: Computer Science - Artificial Intelligence; Computer Science - Learning; Statistics - Machine Learning
• DOI: 10.48550/arxiv.2206.08332

We present BYOL-Explore, a conceptually simple yet general approach for curiosity-driven exploration in visually-complex environments. BYOL-Explore learns a world representation, the world dynamics, and an exploration policy all-together by optimizing a single prediction loss in the latent space with no additional auxiliary objective. We show that BYOL-Explore is effective in DM-HARD-8, a challenging partially-observable continuous-action hard-exploration benchmark with visually-rich 3-D environments. On this benchmark, we solve the majority of the tasks purely through augmenting the extrinsic reward with BYOL-Explore s intrinsic reward, whereas prior work could only get off the ground with human demonstrations. As further evidence of the generality of BYOL-Explore, we show that it achieves superhuman performance on the ten hardest exploration games in Atari while having a much simpler design than other competitive agents.