Rate-optimal estimation for a general class of nonparametric regression models with unknown link functions

TL;DR

This paper introduces a nonparametric regression model generalizing neural networks, capable of estimating additive components and link functions at optimal rates using smoothing splines, applicable to a broad class of models.

Contribution

It proposes a novel estimation approach for a general class of nonparametric models with unknown link functions, achieving rate-optimal convergence.

Findings

Model encompasses neural networks and additive models.

Smoothing splines achieve optimal estimation rates.

Estimation method is applicable to various nonparametric models.

Abstract

This paper discusses a nonparametric regression model that naturally generalizes neural network models. The model is based on a finite number of one-dimensional transformations and can be estimated with a one-dimensional rate of convergence. The model contains the generalized additive model with unknown link function as a special case. For this case, it is shown that the additive components and link function can be estimated with the optimal rate by a smoothing spline that is the solution of a penalized least squares criterion.