Asymptotically Efficient Estimation of Smooth Functionals of Covariance Operators

TL;DR

This paper develops asymptotically efficient estimators for smooth functionals of covariance operators in Hilbert spaces, providing bounds, bias reduction methods, and minimax optimality results.

Contribution

It introduces a new estimator for functionals of covariance operators that achieves asymptotic normality and efficiency, with bias correction techniques in finite-dimensional settings.

Findings

Estimator is asymptotically normal with parametric rate under certain conditions.

Bias of the plug-in estimator can be significant when effective rank is large.

Bias reduction method improves estimator's asymptotic properties in finite-dimensional spaces.

Abstract

Let $X$ be a centered Gaussian random variable in a separable Hilbert space ${\mathbb H}$ with covariance operator $\Sigma.$ We study a problem of estimation of a smooth functional of $\Sigma$ based on a sample $X_1,\dots ,X_n$ of $n$ independent observations of $X.$ More specifically, we are interested in functionals of the form $\langle f(\Sigma), B\rangle,$ where $f:{\mathbb R}\mapsto {\mathbb R}$ is a smooth function and $B$ is a nuclear operator in ${\mathbb H}.$ We prove concentration and normal approximation bounds for plug-in estimator $\langle f(\hat \Sigma),B\rangle,$ $\hat \Sigma:=n^{-1}\sum_{j=1}^n X_j\otimes X_j$ being the sample covariance based on $X_1,\dots, X_n.$ These bounds show that $\langle f(\hat \Sigma),B\rangle$ is an asymptotically normal estimator of its expectation ${\mathbb E}_{\Sigma} \langle f(\hat \Sigma),B\rangle$ (rather than of parameter of interest $\langle f(\Sigma),B\rangle$) with a parametric convergence rate $O(n^{-1/2})$ provided that the effective rank ${\bf r}(\Sigma):= \frac{{\bf tr}(\Sigma)}{\|\Sigma\|}$ (${\rm tr}(\Sigma)$ being the trace and $\|\Sigma\|$ being the operator norm of $\Sigma$) satisfies the assumption ${\bf r}(\Sigma)=o(n).$ At the same time, we show that the bias of this estimator is typically as large as $\frac{{\bf r}(\Sigma)}{n}$ (which is larger than $n^{-1/2}$ if ${\bf r}(\Sigma)\geq n^{1/2}$). In the case when ${\mathbb H}$ is finite-dimensional space of dimension $d=o(n),$ we develop a method of bias reduction and construct an estimator $\langle h(\hat \Sigma),B\rangle$ of $\langle f(\Sigma),B\rangle$ that is asymptotically normal with convergence rate $O(n^{-1/2}).$ Moreover, we study asymptotic properties of the risk of this estimator and prove minimax lower bounds for arbitrary estimators showing the asymptotic efficiency of $\langle h(\hat \Sigma),B\rangle$ in a semi-parametric sense.