Computing Optimal Regularizers for Online Linear Optimization
Khashayar Gatmiry, Jon Schneider, Stefanie Jegelka
TL;DR
This paper introduces an algorithm that finds near-optimal regularizers for online linear optimization, ensuring near-best regret bounds with universal applicability, despite high preprocessing costs.
Contribution
It provides a method to compute regularizers that guarantee near-optimal regret for any convex, symmetric action and loss sets in online linear optimization.
Findings
The algorithm guarantees regret within a constant factor of the best possible.
It runs efficiently online with a membership and linear optimization oracle.
A lower bound shows deciding strong convexity is NP-hard.
Abstract
Follow-the-Regularized-Leader (FTRL) algorithms are a popular class of learning algorithms for online linear optimization (OLO) that guarantee sub-linear regret, but the choice of regularizer can significantly impact dimension-dependent factors in the regret bound. We present an algorithm that takes as input convex and symmetric action sets and loss sets for a specific OLO instance, and outputs a regularizer such that running FTRL with this regularizer guarantees regret within a universal constant factor of the best possible regret bound. In particular, for any choice of (convex, symmetric) action set and loss set we prove that there exists an instantiation of FTRL which achieves regret within a constant factor of the best possible learning algorithm, strengthening the universality result of Srebro et al., 2011. Our algorithm requires preprocessing time and space exponential in the…
Peer Reviews
Decision·ICLR 2025 Conference Withdrawn Submission
1. Technical innovations: The result of the universal regularizer improving the previous result in [1]. The authors also provide construction methods for such universal regularizers.
Some parts of the work are a bit hard to follow. For instance, 1. In Theorem 1, does the author mean $ \sup _{x \in \mathcal{X}}|g(x)|=O\left(\operatorname{Rate}(\mathcal{X}, \mathcal{L})^2\right) $ by $ \sup _{x \in \mathcal{X}}|g|=O\left(\operatorname{Rate}(\mathcal{X}, \mathcal{L})^2\right) $? 2. In Theorem 1, it would be better if the definition or description of the “membership oracle” is introduced first, before stating “given access to a membership oracle”. 3. Also in Theorem 1, what do
1) In terms of the existence of the optimal regularizer, previous studies only prove that there is always an instance of FTRL with $O(Rate(\mathcal{X},\mathcal{L})(\log T)\sqrt{T})$. By contrast, in this paper, the authors show that there exists a regularizer that can reduce the regret of FTRL to $O(Rate(\mathcal{X},\mathcal{L})\sqrt{T})$. 2) The authors propose the first method for computing this ideal regularizer (in an approximated way), which is new to me. Moreover, from the existence of opt
1) Although I agree with the significance of achieving the optimal regret for FTRL in specific OLO problems, the complexity of the proposed method seems too high, i.e., time and space exponential in $d$. Moreover, the authors do not provide any practical instance to show the usefulness of the proposed method, e.g., utilizing the proposed method to compute a regularizer of FTRL in some OLO problems, where people failed to find a regularizer with similar performance. 2) The writing of this paper n
- The authors have presented a detailed survy on related work in online linear optimization and FTRL methods. - This paper is primarily theoretical and provides a complete proof of its findings.
- The writing and organization of this work require further revision. I suggest that the authors summarize their contributions and comparisons with existing methods in a table. Additionally, the authors should clearly outline their method in the form of an algorithm. - I suggest that the author provide a high-level outline of the proof for Theorem 2, as the current version is overly hard to follow and redundant.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Optimization Algorithms Research · Advanced Bandit Algorithms Research · Advanced Wireless Network Optimization
MethodsSparse Evolutionary Training