The Shooting Regressor; Randomized Gradient-Based Ensembles

TL;DR

The paper introduces the Shooting Regressor, an ensemble method using randomized gradient estimators to improve prediction accuracy, with empirical evidence showing it outperforms existing techniques.

Contribution

It presents a novel ensemble approach that combines randomization and gradient information, along with methods to optimize its key parameters.

Findings

Outperforms existing techniques in accuracy

Effective trade-off control between correlation and precision

Empirical validation on a popular dataset

Abstract

An ensemble method is introduced that utilizes randomization and loss function gradients to compute a prediction. Multiple weakly-correlated estimators approximate the gradient at randomly sampled points on the error surface and are aggregated into a final solution. A scaling parameter is described that controls a trade-off between ensemble correlation and precision. Numerical methods for estimating optimal values of the parameter are described. Empirical results are computed over a popular dataset. Inferential statistics on these results show that the method is capable of outperforming existing techniques in terms of increased accuracy.