COBRA: A Combined Regression Strategy

TL;DR

COBRA introduces a fast, model-free method for combining multiple regression estimators by using their collective proximity as a local indicator, achieving asymptotic optimality and strong empirical performance.

Contribution

The paper presents a novel local distance-based approach for combining estimators, outperforming traditional linear or convex combinations in regression tasks.

Findings

Performs asymptotically at least as well as the best combination of estimators.

Demonstrates excellent empirical performance on synthetic and real data.

Offers a fast, model-free alternative with a dedicated R package.

Abstract

A new method for combining several initial estimators of the regression function is introduced. Instead of building a linear or convex optimized combination over a collection of basic estimators $r_1,\dots,r_M$, we use them as a collective indicator of the proximity between the training data and a test observation. This local distance approach is model-free and very fast. More specifically, the resulting nonparametric/nonlinear combined estimator is shown to perform asymptotically at least as well in the $L^2$ sense as the best combination of the basic estimators in the collective. A companion R package called \cobra (standing for COmBined Regression Alternative) is presented (downloadable on \url{http://cran.r-project.org/web/packages/COBRA/index.html}). Substantial numerical evidence is provided on both synthetic and real data sets to assess the excellent performance and velocity of our method in a large variety of prediction problems.