When Analytic Calculus Cracks AdaBoost Code
Jean-Marc Brossier, Olivier Lafitte, Lenny R\'ethor\'e

TL;DR
This paper critically analyzes AdaBoost's implementation, revealing it as a logical combination of weak classifiers rather than an optimization process, and provides an analytical formula for the exact classifier weights.
Contribution
It demonstrates that AdaBoost can be explicitly calculated using a truth table and introduces a system to find the true risk minimum, challenging the conventional understanding of AdaBoost.
Findings
AdaBoost weights can be explicitly derived from a truth table.
The scikit-learn implementation does not follow the original AdaBoost algorithm.
A new method computes the exact risk minimum for AdaBoost classifiers.
Abstract
The principle of boosting in supervised learning involves combining multiple weak classifiers to obtain a stronger classifier. AdaBoost has the reputation to be a perfect example of this approach. This study analyzes the (two classes) AdaBoost procedure implemented in scikit-learn. This paper shows that AdaBoost is an algorithm in name only, as the resulting combination of weak classifiers can be explicitly calculated using a truth table. Indeed, using a logical analysis of the training set with weak classifiers constructing a truth table, we recover, through an analytical formula, the weights of the combination of these weak classifiers obtained by the procedure. We observe that this formula does not give the point of minimum of the risk, we provide a system to compute the exact point of minimum and we check that the AdaBoost procedure in scikit-learn does not implement the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsImbalanced Data Classification Techniques · Adversarial Robustness in Machine Learning · Machine Learning and Algorithms
