On the computation of a non-parametric estimator by convex optimization

TL;DR

This paper introduces a new, computationally efficient algorithm for non-parametric estimation of linear functionals using convex optimization, making the method more accessible for high-dimensional problems.

Contribution

It presents an alternative algorithm that leverages existing optimization software, reducing computational costs in high-dimensional settings for non-parametric estimators.

Findings

Algorithm is easier to implement with existing software.

Significantly reduces computational expense in high-dimensional spaces.

Facilitates wider adoption of the estimation technique.

Abstract

Estimation of linear functionals from observed data is an important task in many subjects. Juditsky & Nemirovski [The Annals of Statistics 37.5A (2009): 2278-2300] propose a framework for non-parametric estimation of linear functionals in a very general setting, with nearly minimax optimal confidence intervals. They compute this estimator and the associated confidence interval by approximating the saddle-point of a function. While this optimization problem is convex, it is rather difficult to solve using existing off-the-shelf optimization software. Furthermore, this computation can be expensive when the estimators live in a high-dimensional space. We propose a different algorithm to construct this estimator. Our algorithm can be used with existing optimization software and is much cheaper to implement even when the estimators are in a high-dimensional space, as long as the Hellinger affinity (or the Bhattacharyya coefficient) for the chosen parametric distribution can be efficiently computed given the parameters. We hope that our algorithm will foster the adoption of this estimation technique to a wider variety of problems with relative ease.