Online Learning with Low Rank Experts
Elad Hazan, Tomer Koren, Roi Livni, Yishay Mansour

TL;DR
This paper addresses online learning with expert advice when experts' losses lie in a low-dimensional subspace, providing algorithms with regret bounds that depend only on the subspace dimension, not the total number of experts.
Contribution
It introduces algorithms with regret bounds independent of the number of experts, depending only on the low-rank structure, and establishes tight bounds for stochastic and adversarial models.
Findings
Regret bound of Θ(√(dT)) in stochastic setting.
Regret bounds of O(d√T) upper and Ω(√(dT)) lower in adversarial setting.
Algorithms effectively leverage low-rank structure for improved regret bounds.
Abstract
We consider the problem of prediction with expert advice when the losses of the experts have low-dimensional structure: they are restricted to an unknown -dimensional subspace. We devise algorithms with regret bounds that are independent of the number of experts and depend only on the rank . For the stochastic model we show a tight bound of , and extend it to a setting of an approximate subspace. For the adversarial model we show an upper bound of and a lower bound of .
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Bandit Algorithms Research · Machine Learning and Algorithms · Stochastic Gradient Optimization Techniques
