Limit Cycles of AdaBoost

TL;DR

This paper investigates the cycling behavior of AdaBoost, modeling it as a dynamical system and establishing a connection with continued fractions to explain its periodic states.

Contribution

It introduces a novel dynamical systems perspective on AdaBoost's cycling behavior and links it to continued fractions, providing a new theoretical understanding.

Findings

AdaBoost's cycling behavior can be modeled as a dynamical map.

A correspondence between cycling and continued fractions is established.

The paper offers a self-contained explanation for AdaBoost's periodic states.

Abstract

The iterative weight update for the AdaBoost machine learning algorithm may be realized as a dynamical map on a probability simplex. When learning a low-dimensional data set this algorithm has a tendency towards cycling behavior, which is the topic of this paper. AdaBoost's cycling behavior lends itself to direct computational methods that are ineffective in the general, non-cycling case of the algorithm. From these computational properties we give a concrete correspondence between AdaBoost's cycling behavior and continued fractions dynamics. Then we explore the results of this correspondence to expound on how the algorithm comes to be in this periodic state at all. What we intend for this work is to be a novel and self-contained explanation for the cycling dynamics of this machine learning algorithm.