Stability via resampling: statistical problems beyond the real line

TL;DR

This paper establishes a finite-sample theoretical guarantee for the stability of bagging with resampling methods across general output spaces, supported by empirical evaluations in various statistical contexts.

Contribution

It extends stability guarantees of bagging to algorithms with outputs beyond real numbers, covering a broad range of statistical applications.

Findings

Bagging is stable in general output spaces under the proposed theoretical conditions.

Empirical results confirm stability across diverse data types and problem settings.

Resampling methods improve the robustness of statistical algorithms in practice.

Abstract

Model averaging techniques based on resampling methods (such as bootstrapping or subsampling) have been utilized across many areas of statistics, often with the explicit goal of promoting stability in the resulting output. We provide a general, finite-sample theoretical result guaranteeing the stability of bagging when applied to algorithms that return outputs in a general space, so that the output is not necessarily a real-valued -- for example, an algorithm that estimates a vector of weights or a density function. We empirically assess the stability of bagging on synthetic and real-world data for a range of problem settings, including causal inference, nonparametric regression, and Bayesian model selection.