A Kernel-based Consensual Aggregation for Regression

TL;DR

This paper presents a flexible kernel-based aggregation method for regression that combines estimators using kernel-defined weights, ensuring consistency and improving performance with optimized gradient descent estimation.

Contribution

It extends previous aggregation frameworks to a kernel-based approach, demonstrating consistency and efficiency improvements through gradient descent optimization.

Findings

Kernel-based aggregation inherits estimator consistency.

Gradient descent accelerates parameter estimation.

Smoother kernels improve regression performance.

Abstract

In this article, we introduce a kernel-based consensual aggregation method for regression problems. We aim to flexibly combine individual regression estimators $r_1, r_2, \ldots, r_M$ using a weighted average where the weights are defined based on some kernel function to build a target prediction. This work extends the context of Biau et al. (2016) to a more general kernel-based framework. We show that this more general configuration also inherits the consistency of the basic consistent estimators. Moreover, an optimization method based on gradient descent algorithm is proposed to efficiently and rapidly estimate the key parameter of the strategy. The numerical experiments carried out on several simulated and real datasets are also provided to illustrate the speed-up of gradient descent algorithm in estimating the key parameter and the improvement of overall performance of the method with the introduction of smoother kernel functions.