Random matrices based schemes for stable and robust nonparametric and functional regression estimators

TL;DR

This paper introduces a novel, stable, and robust nonparametric regression estimator based on random matrix pseudo-inverses and Jacobi polynomials, extending to functional regression with demonstrated effectiveness.

Contribution

It presents a new nonparametric regression scheme using random matrices and extends it to linear functional regression, avoiding regularization and working under various sampling distributions.

Findings

The estimator achieves accurate $L_2$ and $L_2$-risk errors.

It is stable, robust, and computationally efficient.

Numerical simulations and real data validate the approach.

Abstract

In the first part of this work, we develop a novel scheme for solving nonparametric regression problems. That is the approximation of possibly low regular and noised functions from the knowledge of their approximate values given at some random points. Our proposed scheme is based on the use of the pseudo-inverse of a random projection matrix, combined with some specific properties of the Jacobi polynomials system, as well as some properties of positive definite random matrices. This scheme has the advantages to be stable, robust, accurate and fairly fast in terms of execution time. In particular, we provide an $L_2$ as well as an $L_2-$risk errors of our proposed nonparametric regression estimator. Moreover and unlike most of the existing nonparametric regression estimators, no extra regularization step is required by our proposed estimator. Although, this estimator is initially designed to work with random sampling set of uni-variate i.i.d. random variables following a Beta distribution, we show that it is still works for a wide range of sampling distribution laws. Moreover, we briefly describe how our estimator can be adapted in order to handle the multivariate case of random sampling sets. In the second part of this work, we extend the random pseudo-inverse scheme technique to build a stable and accurate estimator for solving linear functional regression (LFR) problems. A dyadic decomposition approach is used to construct this last stable estimator for the LFR problem. Alaso, we give an $L_2-$risk error of our proposed LFR estimator. Finally, the performance of the two proposed estimators are illustrated by various numerical simulations. In particular, a real dataset is used to illustrate the performance of our nonparametric regression estimator.