Adaptive estimation in regression and complexity of approximation of random fields

TL;DR

This thesis develops adaptive nonparametric regression methods robust to noise misspecification and analyzes the complexity of approximating high-dimensional random fields, providing theoretical bounds and insights into high-dimensional approximation challenges.

Contribution

It introduces a robust adaptive estimation procedure with relaxed propagation conditions and analyzes the asymptotic complexity of approximating high-dimensional random fields.

Findings

Adaptive estimation tolerates covariance misspecification of order 1/ log(n).

Oracle risk bounds are established for the estimation method.

Complexity analysis reveals how approximation difficulty scales with dimension.

Abstract

In this thesis we study adaptive nonparametric regression with noise misspecification and the complexity of approximation of random fields in dependence of the dimension. First, we consider the problem of pointwise estimation in nonparametric regression with heteroscedastic additive Gaussian noise. We use the method of local approximation applying the Lepski method for selecting one estimate from the set of linear estimates obtained by the different degrees of localization. This approach is combined with the "propagation conditions" on the choice of critical values of the procedure, as suggested recently by Spokoiny and Vial [Ann.Stat., 2009]. The "propagation conditions" are relaxed for the model with misspecified covariance structure. We show that this procedure allows a misspecification of the covariance matrix with a relative error of order 1/ log(n), where n is the sample size. The quality of estimation is measured in terms of "oracle" risk bounds. We then turn to the approximation of d-parametric random fields of tensor product-type by means of n-term partial sums of the Karhunen-Lo\`eve expansion. The analysis is restricted to the average case setting. The quantity of interest is the information complexity describing the minimal number of terms in the partial sums, which guarantees an error not exceeding a given level. The behavior of this quantity when the dimension goes to infinity is the subject of our study.