Nonparametric estimation by convex programming

TL;DR

This paper introduces computationally efficient, minimax optimal nonparametric estimation methods using convex programming for problems involving convex sets, affine mappings, and parametric density families, with applications to estimating linear forms and recovering the underlying parameter.

Contribution

It develops novel convex programming-based estimation routines that are minimax optimal for a broad class of nonparametric problems without extra assumptions.

Findings

Estimation routines are minimax optimal within a constant factor.

Methods are computationally efficient for several density families.

Applications include recovering the parameter vector in Euclidean norm.

Abstract

The problem we concentrate on is as follows: given (1) a convex compact set $X$ in ${\mathbb{R}}^n$, an affine mapping $x\mapsto A(x)$, a parametric family $\{p_{\mu}(\cdot)\}$ of probability densities and (2) $N$ i.i.d. observations of the random variable $\omega$, distributed with the density $p_{A(x)}(\cdot)$ for some (unknown) $x\in X$, estimate the value $g^Tx$ of a given linear form at $x$. For several families $\{p_{\mu}(\cdot)\}$ with no additional assumptions on $X$ and $A$, we develop computationally efficient estimation routines which are minimax optimal, within an absolute constant factor. We then apply these routines to recovering $x$ itself in the Euclidean norm.