← Back to papers
2007 NIPS NeurIPS 2007

Fitted Q-iteration in continuous action-space MDPs

Abstract

We consider continuous state, continuous action batch reinforcement learning where the goal is to learn a good policy from a sufficiently rich trajectory generated by another policy. We study a variant of fitted Q-iteration, where the greedy action selection is replaced by searching for a policy in a restricted set of candidate policies by maximizing the average action values. We provide a rigorous theoretical analysis of this algorithm, proving what we believe is the first finite-time bounds for value-function based algorithms for continuous state- and action-space problems.

📈 Trend Setter — Offline RL
🧭 Keyword Pioneer — fitted q-iteration
🐣 Hot Topic Early Bird — reinforcement learning
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics
🌉 Interdisciplinary Bridge — Machine Learning and Reinforcement Learning

Authors

András Antos , Csaba Szepesvári , Rémi Munos

Topics

Reinforcement Learning > Methods > Deep RL Reinforcement Learning > Methods > Offline RL Reinforcement Learning > Methods > Policy Learning Machine Learning > Learning Types > Reinforcement Learning

Keywords

reinforcement learning markov decision process value function continuous control fitted q-iteration batch learning value function approximation batch reinforcement learning continuous action space
Download PDF

Related papers

Exponential Family Predictive Representations of State 2007
Privacy-Preserving Belief Propagation and Sampling 2007
Efficient Principled Learning of Thin Junction Trees 2007
How SVMs can estimate quantiles and the median 2007
Rapid Inference on a Novel AND/OR graph for Object Detection, Segmentation and Parsing 2007