Star algorithm for NN ensembling

TL;DR

This paper introduces a novel neural network ensembling algorithm based on Audibert's empirical star algorithm, providing theoretical bounds and empirical comparisons to existing methods for improved model performance.

Contribution

The paper presents a new ensembling algorithm with proven optimal minimax bounds and empirical validation across regression and classification tasks.

Findings

Achieves optimal theoretical minimax bounds on excess squared risk.

Performs competitively with or better than popular ensembling methods.

Validated through empirical studies on regression and classification datasets.

Abstract

Neural network ensembling is a common and robust way to increase model efficiency. In this paper, we propose a new neural network ensemble algorithm based on Audibert's empirical star algorithm. We provide optimal theoretical minimax bound on the excess squared risk. Additionally, we empirically study this algorithm on regression and classification tasks and compare it to most popular ensembling methods.