Mean and Variance Estimation Complexity in Arbitrary Distributions via Wasserstein Minimization

TL;DR

This paper investigates the complexity of estimating mean and variance parameters in arbitrary distributions, showing that while MLE is NP-hard, Wasserstein minimization allows efficient approximation.

Contribution

It demonstrates that Wasserstein distance enables polynomial-time approximations for mean and variance estimation in complex distributions, despite MLE's NP-hardness.

Findings

MLE estimation is NP-hard for these parameters.

Wasserstein minimization provides polynomial-time $ ext{poly}(1/\varepsilon)$ approximations.

The approach applies to arbitrary distributions with known density $f_0$.

Abstract

Parameter estimation is a fundamental challenge in machine learning, crucial for tasks such as neural network weight fitting and Bayesian inference. This paper focuses on the complexity of estimating translation $\boldsymbol{\mu} \in \mathbb{R}^l$ and shrinkage $\sigma \in \mathbb{R}_{++}$ parameters for a distribution of the form $\frac{1}{\sigma^l} f_0 \left( \frac{\boldsymbol{x} - \boldsymbol{\mu}}{\sigma} \right)$, where $f_0$ is a known density in $\mathbb{R}^l$ given $n$ samples. We highlight that while the problem is NP-hard for Maximum Likelihood Estimation (MLE), it is possible to obtain $\varepsilon$-approximations for arbitrary $\varepsilon > 0$ within $\text{poly} \left( \frac{1}{\varepsilon} \right)$ time using the Wasserstein distance.