← Back to papers
2019 NIPS NeurIPS 2019

Regret Minimization for Reinforcement Learning with Vectorial Feedback and Complex Objectives

Abstract

We consider an agent who is involved in an online Markov decision process, and receives a vector of outcomes every round. The agent aims to simultaneously optimize multiple objectives associated with the multi-dimensional outcomes. Due to state transitions, it is challenging to balance the vectorial outcomes for achieving near-optimality. In particular, contrary to the single objective case, stationary policies are generally sub-optimal. We propose a no-regret algorithm based on the Frank-Wolfe algorithm (Frank and Wolfe 1956), UCRL2 (Jaksch et al. 2010), as well as a crucial and novel gradient threshold procedure. The procedure involves carefully delaying gradient updates, and returns a non-stationary policy that diversifies the outcomes for optimizing the objectives.

🌉 Interdisciplinary Bridge — Machine Learning and Mathematics & Optimization and Reinforcement Learning
📈 Trend Setter — Multi-Objective Optimization
🧭 Keyword Pioneer — vectorial feedback
🐣 Hot Topic Early Bird — multi-objective optimization
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio

Authors

Wang Chi Cheung

Topics

Machine Learning > Optimization & Theory > Optimization Reinforcement Learning > Methods > Deep RL Machine Learning > Learning Types > Reinforcement Learning Mathematics & Optimization > Optimization > Game Theory Machine Learning > Learning Types > Multi-Objective Optimization

Keywords

reinforcement learning markov decision process regret minimization multi-objective optimization frank-wolfe algorithm vectorial feedback
Download PDF

Related papers

Two Generator Game: Learning to Sample via Linear Goodness-of-Fit Test 2019
Metalearned Neural Memory 2019
Model Similarity Mitigates Test Set Overuse 2019
Continual Unsupervised Representation Learning 2019
Reinforcement Learning with Convex Constraints 2019