$(\alpha,\beta)$-Stability for Boosting Vector-Valued Prediction

TL;DR

This paper introduces a new stability property called $( ext{ extalpha}, ext{eta})$-stability for vector-valued boosting, providing a theoretical framework that enhances understanding and guarantees in structured prediction tasks.

Contribution

It formalizes $( ext{ extalpha}, ext{eta})$-stability for vector-valued prediction and develops a boosting framework supported by this property, with theoretical guarantees.

Findings

Characterizes stability under various divergences like $ ext{ extl}_1$, $ ext{ extl}_2$, TV, Hellinger, and KL.

Proposes the ext{ extgeomedboost} framework with exponential error decay.

Provides generalization bounds based on stability and weak learner conditions.

Abstract

Despite the widespread use of boosting in structured prediction, a general theoretical understanding of aggregation beyond scalar prediction remains incomplete. We study vector-valued prediction under a target divergence and identify a geometric stability property under which aggregation amplifies weak guarantees into strong ones. We formalize this property as $(\alpha,\beta)$-stability by geometric median and show how it supports a boosting framework based on exponential reweighting and geometric-median aggregation. For vector-valued prediction, we characterize this stability property under several natural divergences: $\ell_1$ and $\ell_2$ distances for unconstrained vector-valued prediction, and TV, Hellinger, and KL for density estimation over finite probability vectors. Building on these results, we propose a generic boosting framework \geomedboost. Under a weak learner condition and $(\alpha,\beta)$-stability, we obtain exponential decay of the empirical divergence error, which then yields population guarantees through a generalization bound.