← Back to papers
2017 ICML ICML 2017

DARLA: Improving Zero-Shot Transfer in Reinforcement Learning

Abstract

Domain adaptation is an important open problem in deep reinforcement learning (RL). In many scenarios of interest data is hard to obtain, so agents may learn a source policy in a setting where data is readily available, with the hope that it generalises well to the target domain. We propose a new multi-stage RL agent, DARLA (DisentAngled Representation Learning Agent), which learns to see before learning to act. DARLA’s vision is based on learning a disentangled representation of the observed environment. Once DARLA can see, it is able to acquire source policies that are robust to many domain shifts – even with no access to the target domain. DARLA significantly outperforms conventional baselines in zero-shot domain adaptation scenarios, an effect that holds across a variety of RL environments (Jaco arm, DeepMind Lab) and base RL algorithms (DQN, A3C and EC).

🌱 Topic Pioneer β€” Zero-Shot Learning
πŸŒ‰ Interdisciplinary Bridge β€” Artificial Intelligence and Machine Learning
πŸ“ˆ Trend Setter β€” Zero-Shot Learning
🧭 Keyword Pioneer β€” zero-shot transfer
🐣 Hot Topic Early Bird β€” deep reinforcement learning
🐝 Cross-Pollinator β€” Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Speech & Audio

Authors

Irina Higgins , Arka Pal , Andrei Rusu , Loic Matthey , Christopher Burgess , Alexander Pritzel , Matthew Botvinick , Charles Blundell , Alexander Lerchner

Topics

Machine Learning > Core Methods > Representation Learning Machine Learning > Application Areas > Domain Adaptation Reinforcement Learning > Methods > Deep RL Artificial Intelligence > Learning Paradigms > Zero-Shot Learning Machine Learning > Learning Types > Transfer Learning Machine Learning > Learning Paradigms > Zero-Shot Learning

Keywords

deep reinforcement learning reinforcement learning domain adaptation disentangled representation zero-shot transfer domain shift disentangled representation learning source policy zero-shot domain adaptation
Download PDF

Related papers

Bottleneck Conditional Density Estimation 2017
Constrained Policy Optimization 2017
Near-Optimal Design of Experiments via Regret Minimization 2017
Input Convex Neural Networks 2017
An Efficient, Sparsity-Preserving, Online Algorithm for Low-Rank Approximation 2017