Gradient COBRA: A kernel-based consensual aggregation for regression

TL;DR

Gradient COBRA introduces a kernel-based aggregation method for regression that flexibly combines estimators, ensuring consistency and efficiency, with demonstrated accuracy on simulated and real datasets, including domain adaptation scenarios.

Contribution

This work extends the COBRA aggregation framework to a kernel-based approach, providing theoretical guarantees and an efficient gradient descent optimization method.

Findings

Achieves the same convergence rate as classical COBRA.

Demonstrates high accuracy on simulated and real datasets.

Shows domain adaptation-like properties in physics data.

Abstract

In this article, we introduce a kernel-based consensual aggregation method for regression problems. We aim to exibly combine individual regression estimators $r_1, \ldots, r_M$ using a weighted average where the weights are dened based on predicted features given by all the basic estimators and some kernel function. This work extends the context of Biau et al. (2016) to a more general kernel-based framework. We show that this more general conguration also inherits the consistency of the basic consistent estimators, and the same convergence rate as in the classical method is achieved. Moreover, an optimization method based on gradient descent algorithm is proposed to eciently and rapidly estimate the key parameter of the strategy. Various numerical experiments carried out on several simulated and real datasets are also provided to illustrate the eciency and accuracy of the proposed method. Moreover, a domain adaptation-like property of the aggregation strategy is also illustrated on a physics data provided by Commissariat {\`a} l'{\'E}nergie Atomique (CEA).