Learning Entangled Single-Sample Gaussians in the Subset-of-Signals Model

TL;DR

This paper investigates mean estimation in entangled single-sample Gaussian models with a subset of variances bounded, proposing an iterative averaging method and establishing matching upper and lower bounds for the estimation error.

Contribution

The paper introduces a subset-of-signals model for entangled single-sample Gaussians, analyzes a simple averaging method, and derives tight bounds extending previous results.

Findings

Proposed an iterative averaging method achieving error $O(\frac{\sqrt{n\ln n}}{m})$ for $m=\Omega(\sqrt{n\ln n})$.

Established lower bounds showing the error cannot be smaller than certain functions of $n$ and $m$ in various regimes.

Extended the understanding of error bounds for mean estimation in entangled single-sample Gaussian models.

Abstract

In the setting of entangled single-sample distributions, the goal is to estimate some common parameter shared by a family of $n$ distributions, given one single sample from each distribution. This paper studies mean estimation for entangled single-sample Gaussians that have a common mean but different unknown variances. We propose the subset-of-signals model where an unknown subset of $m$ variances are bounded by 1 while there are no assumptions on the other variances. In this model, we analyze a simple and natural method based on iteratively averaging the truncated samples, and show that the method achieves error $O \left(\frac{\sqrt{n\ln n}}{m}\right)$ with high probability when $m=\Omega(\sqrt{n\ln n})$, matching existing bounds for this range of $m$. We further prove lower bounds, showing that the error is $\Omega\left(\left(\frac{n}{m^4}\right)^{1/2}\right)$ when $m$ is between $\Omega(\ln n)$ and $O(n^{1/4})$, and the error is $\Omega\left(\left(\frac{n}{m^4}\right)^{1/6}\right)$ when $m$ is between $\Omega(n^{1/4})$ and $O(n^{1 - \epsilon})$ for an arbitrarily small $\epsilon>0$, improving existing lower bounds and extending to a wider range of $m$.