Representation of small ball probabilities in Hilbert space and lower bound in regression for functional data

TL;DR

This paper investigates small ball probabilities in Hilbert spaces and uses these results to establish lower bounds for nonparametric regression rates with functional data, revealing logarithmic convergence limits.

Contribution

It characterizes small ball probabilities as Gamma-varying functions and applies this to derive minimax lower bounds in functional data regression.

Findings

Small ball probabilities belong to the class of Gamma-varying functions.

Nonparametric regression risk bounds are of order (log n)^(-τ).

Polynomial rates are not achievable in this setting.

Abstract

Let $S=\sum_{i=1}^{+\infty}\lambda_{i}Z_{i}$ where the $Z_{i}$'s are i.d.d. positive with $\mathbb{E}\| Z\| ^{3}<+\infty$ and $(\lambda_{i})_{i\in\mathbb{N}}$ a positive nonincreasing sequence such that $\sum\lambda_{i}<+\infty$. We study the small ball probability $\mathbb{P}(S<\epsilon) $ when $\epsilon\downarrow0$. We start from a result by Lifshits (1997) who computed this probability by means of the Laplace transform of $S$. We prove that $\mathbb{P}(S<\cdot) $ belongs to a class of functions introduced by de Haan, well-known in extreme value theory, the class of Gamma-varying functions, for which an exponential-integral representation is available. This approach allows to derive bounds for the rate in nonparametric regression for functional data at a fixed point $x_{0}$ : $\mathbb{E}(y|X=x_{0}%) $ where $(y_{i},X_{i})_{1\leq i\leq n}$ is a sample in $(\mathbb{R},\mathcal{F}) $ and $\mathcal{F}$ is some space of functions. It turns out that, in a general framework, the minimax lower bound for the risk is of order $(\log n)^{-\tau}$ for some $\tau>0$ depending on the regularity of the data and polynomial rates cannot be achieved.