On sample complexity for covariance estimation via the unadjusted Langevin algorithm

TL;DR

This paper provides theoretical guarantees on the number of samples needed for covariance estimation using the unadjusted Langevin algorithm, highlighting efficiency differences between single-chain and parallel approaches.

Contribution

It introduces new sample complexity bounds for covariance estimation with ULA and compares single-chain versus parallel implementations, emphasizing bias reduction effects.

Findings

Single-chain ULA has lower sample complexity than embarrassingly parallel ULA by a logarithmic factor.

A concentration bound for the sample covariance matrix is derived using a log-Sobolev inequality.

The results quantify the efficiency of ULA in covariance estimation for strongly log-concave distributions.

Abstract

We establish sample complexity guarantees for estimating the covariance matrix of a strongly log-concave smooth distribution using the unadjusted Langevin algorithm (ULA). We quantitatively compare our complexity estimates on single-chain ULA with embarrassingly parallel ULA and derive that the sample complexity of the single-chain approach is smaller than that of embarrassingly parallel ULA by a logarithmic factor in the dimension and the reciprocal of the prescribed precision, with the difference arising from effective bias reduction through burn-in. The key technical contribution is a concentration bound for the sample covariance matrix around its expectation, derived via a log-Sobolev inequality for the joint distribution of ULA iterates.