Estimation of Smooth Functionals in Normal Models: Bias Reduction and Asymptotic Efficiency

TL;DR

This paper develops bias-reduced estimators for smooth functionals of normal distribution parameters, achieving minimax optimal rates and asymptotic efficiency in high-dimensional settings with spectrum constraints.

Contribution

It introduces a novel bias reduction technique for estimating smooth functionals in normal models, extending to high-dimensional cases with spectral constraints.

Findings

Estimator achieves minimax optimal error rates.

Estimator is asymptotically efficient under certain smoothness conditions.

Method applies to more general models beyond normal distributions.

Abstract

Let $X_1,\dots, X_n$ be i.i.d. random variables sampled from a normal distribution $N(\mu,\Sigma)$ in ${\mathbb R}^d$ with unknown parameter $\theta=(\mu,\Sigma)\in \Theta:={\mathbb R}^d\times {\mathcal C}_+^d,$ where ${\mathcal C}_+^d$ is the cone of positively definite covariance operators in ${\mathbb R}^d.$ Given a smooth functional $f:\Theta \mapsto {\mathbb R}^1,$ the goal is to estimate $f(\theta)$ based on $X_1,\dots, X_n.$ Let $$ \Theta(a;d):={\mathbb R}^d\times \Bigl\{\Sigma\in {\mathcal C}_+^d: \sigma(\Sigma)\subset [1/a, a]\Bigr\}, a\geq 1, $$ where $\sigma(\Sigma)$ is the spectrum of covariance $\Sigma.$ Let $\hat \theta:=(\hat \mu, \hat \Sigma),$ where $\hat \mu$ is the sample mean and $\hat \Sigma$ is the sample covariance, based on the observations $X_1,\dots, X_n.$ For an arbitrary functional $f\in C^s(\Theta),$ $s=k+1+\rho, k\geq 0, \rho\in (0,1],$ we define a functional $f_k:\Theta \mapsto {\mathbb R}$ such that \begin{align*} & \sup_{\theta\in \Theta(a;d)}\|f_k(\hat \theta)-f(\theta)\|_{L_2({\mathbb P}_{\theta})} \lesssim_{s, \beta} \|f\|_{C^{s}(\Theta)} \biggr[\biggl(\frac{a}{\sqrt{n}} \bigvee a^{\beta s}\biggl(\sqrt{\frac{d}{n}}\biggr)^{s} \biggr)\wedge 1\biggr], \end{align*} where $\beta =1$ for $k=0$ and $\beta>s-1$ is arbitrary for $k\geq 1.$ This error rate is minimax optimal and similar bounds hold for more general loss functions. If $d=d_n\leq n^{\alpha}$ for some $\alpha\in (0,1)$ and $s\geq \frac{1}{1-\alpha},$ the rate becomes $O(n^{-1/2}).$ Moreover, for $s>\frac{1}{1-\alpha},$ the estimators $f_k(\hat \theta)$ is shown to be asymptotically efficient. The crucial part of the construction of estimator $f_k(\hat \theta)$ is a bias reduction method studied in the paper for more general statistical models than normal.