← Back to papers
2020 JMLR JMLR 2020

Contextual Bandits with Continuous Actions: Smoothing, Zooming, and Adapting

Abstract

We study contextual bandit learning with an abstract policy class and continuous action space. We obtain two qualitatively different regret bounds: one competes with a smoothed version of the policy class under no continuity assumptions, while the other requires standard Lipschitz assumptions. Both bounds exhibit data-dependent “zooming” behavior and, with no tuning, yield improved guarantees for benign problems. We also study adapting to unknown smoothness parameters, establishing a price-of-adaptivity and deriving optimal adaptive algorithms that require no additional information. [abs] [ pdf ][ bib ] © JMLR 2020. (edit, beta)

🌉 Interdisciplinary Bridge — Artificial Intelligence and Machine Learning and Mathematics & Optimization
🧭 Keyword Pioneer — zooming algorithm
🐝 Cross-Pollinator — Artificial Intelligence, Data Science & Analytics, Deep Learning, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy

Authors

Akshay Krishnamurthy , John Langford , Aleksandrs Slivkins , Chicheng Zhang

Topics

Artificial Intelligence > Learning Paradigms > Few-Shot Learning Machine Learning > Learning Types > Active Learning Reinforcement Learning > Methods Mathematics & Optimization > Optimization > Online Algorithms Machine Learning > Learning Types > Multi-Armed Bandits

Keywords

regret bound online algorithm contextual bandit continuous action zooming algorithm lipschitz assumption
Download PDF

Related papers

Weighted Message Passing and Minimum Energy Flow for Heterogeneous Stochastic Block Models with Side Information 2020
Connecting Spectral Clustering to Maximum Margins and Level Sets 2020
Stochastic Nested Variance Reduction for Nonconvex Optimization 2020
Gradient Descent for Sparse Rank-One Matrix Completion for Crowd-Sourced Aggregation of Sparsely Interacting Workers 2020
A Sparse Semismooth Newton Based Proximal Majorization-Minimization Algorithm for Nonconvex Square-Root-Loss Regression Problems 2020