Empirical Wavelet-based Estimation for Non-linear Additive Regression Models

TL;DR

This paper introduces a data-driven wavelet-based estimator for non-linear additive regression models with random designs, providing theoretical guarantees and demonstrating practical effectiveness through simulations.

Contribution

It proposes a novel orthogonal projection estimator using empirical wavelet coefficients for additive models with random designs, without requiring equispaced predictor data.

Findings

The estimator is mean-square consistent under certain smoothness conditions.

The method achieves favorable convergence rates in low-dimensional settings.

Practical results from simulations demonstrate its applicability.

Abstract

Additive regression models are actively researched in the statistical field because of their usefulness in the analysis of responses determined by non-linear relationships with multivariate predictors. In this kind of statistical models, the response depends linearly on unknown functions of predictor variables and typically, the goal of the analysis is to make inference about these functions. In this paper, we consider the problem of Additive Regression with random designs from a novel viewpoint: we propose an estimator based on an orthogonal projection onto a multiresolution space using empirical wavelet coefficients that are fully data driven. In this setting, we derive a mean-square consistent estimator based on periodic wavelets on the interval $[0,1]$. For construction of the estimator, we assume that the joint distribution of predictors is non-zero and bounded on its support; We also assume that the functions belong to a Sobolev space and integrate to zero over the $[0,1]$ interval, which guarantees model identifiability and convergence of the proposed method. Moreover, we provide the $\mathbb{L}_{2}$ risk analysis of the estimator and derive its convergence rate. Theoretically, we show that this approach achieves good convergence rates when the dimensionality of the problem is relatively low and the set of unknown functions is sufficiently smooth. In this approach, the results are obtained without the assumption of an equispaced design, a condition that is typically assumed in most wavelet-based procedures. Finally, we show practical results obtained from simulated data, demonstrating the potential applicability of our method in the problem of additive regression models with random designs.