Optimal AdaBoost Converges

TL;DR

This paper provides formal proofs demonstrating that AdaBoost's classifier and margins converge to values consistent with longstanding theoretical predictions, enhancing understanding of its convergence behavior.

Contribution

It offers the first comprehensive formal proofs of AdaBoost's convergence properties for classifiers and margins, confirming longstanding conjectures.

Findings

AdaBoost's classifier converges to a stable solution.

Margins of AdaBoost also converge to a specific value.

Various related quantities associated with the classifier converge as well.

Abstract

The following work is a preprint collection of formal proofs regarding the convergence properties of the AdaBoost machine learning algorithm's classifier and margins. Various math and computer science papers have been written regarding conjectures and special cases of these convergence properties. Furthermore, the margins of AdaBoost feature prominently in the research surrounding the algorithm. At the zenith of this paper we present how AdaBoost's classifier and margins converge on a value that agrees with decades of research. After this, we show how various quantities associated with the combined classifier converge.