Boosting as Frank-Wolfe
Ryotaro Mitsuboshi, Kohei Hatano, Eiji Takimoto

TL;DR
This paper introduces a unified boosting framework based on the Frank-Wolfe algorithm, which generalizes existing boosting methods for soft margin optimization and improves practical performance by combining secondary algorithms.
Contribution
It presents a generic boosting scheme that unifies and extends LPBoost, ERLPBoost, and C-ERLPBoost using the Frank-Wolfe algorithm, allowing flexible incorporation of secondary algorithms.
Findings
The scheme retains convergence guarantees similar to existing methods.
Experiments show the scheme performs comparably to LPBoost in practice.
The framework offers a unified view of boosting algorithms for soft margin problems.
Abstract
Some boosting algorithms, such as LPBoost, ERLPBoost, and C-ERLPBoost, aim to solve the soft margin optimization problem with the -norm regularization. LPBoost rapidly converges to an -approximate solution in practice, but it is known to take iterations in the worst case, where is the sample size. On the other hand, ERLPBoost and C-ERLPBoost are guaranteed to converge to an -approximate solution in iterations. However, the computation per iteration is very high compared to LPBoost. To address this issue, we propose a generic boosting scheme that combines the Frank-Wolfe algorithm and any secondary algorithm and switches one to the other iteratively. We show that the scheme retains the same convergence guarantee as ERLPBoost and C-ERLPBoost. One can incorporate any secondary algorithm to improve in…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSparse and Compressive Sensing Techniques · Machine Learning and Algorithms · Stochastic Gradient Optimization Techniques
