← Back to papers
2022 NIPS NeurIPS 2022

BYOL-Explore: Exploration by Bootstrapped Prediction

Abstract

We present BYOL-Explore, a conceptually simple yet general approach for curiosity-driven exploration in visually complex environments. BYOL-Explore learns the world representation, the world dynamics and the exploration policy all-together by optimizing a single prediction loss in the latent space with no additional auxiliary objective. We show that BYOL-Explore is effective in DM-HARD-8, a challenging partially-observable continuous-action hard-exploration benchmark with visually rich 3-D environment. On this benchmark, we solve the majority of the tasks purely through augmenting the extrinsic reward with BYOL-Explore intrinsic reward, whereas prior work could only get off the ground with human demonstrations. As further evidence of the generality of BYOL-Explore, we show that it achieves superhuman performance on the ten hardest exploration games in Atari while having a much simpler design than other competitive agents.

🌉 Interdisciplinary Bridge — Artificial Intelligence and Deep Learning and Machine Learning and Reinforcement Learning
🧭 Keyword Pioneer — bootstrapped prediction
🐣 Hot Topic Early Bird — world model
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio

Authors

Zhaohan Guo , Shantanu Thakoor , Miruna Pislar , Bernardo Avila Pires , Florent Altché , Corentin Tallec , Alaa Saade , Daniele Calandriello , Jean-Bastien Grill , Yunhao Tang , Michal Valko , Rémi Munos , Mohammad Gheshlaghi azar , Bilal Piot

Topics

Machine Learning > Learning Types > Self-Supervised Learning Reinforcement Learning > Methods > Deep RL Deep Learning > Learning Types > Self-Supervised Learning Artificial Intelligence > Core AI > Reinforcement Learning

Keywords

reinforcement learning self-supervised learning curiosity-driven learning world model intrinsic reward curiosity-driven exploration prediction loss world representation bootstrapped prediction
Download PDF

Related papers

Transferring Pre-trained Multimodal Representations with Cross-modal Similarity Matching 2022
A Theoretical View on Sparsely Activated Networks 2022
Prune and distill: similar reformatting of image information along rat visual cortex and deep neural networks 2022
Matryoshka Representation Learning 2022
Off-Policy Evaluation with Deficient Support Using Side Information 2022