Sample Complexity Bounds for Estimating Probability Divergences under Invariances

TL;DR

This paper investigates how invariances under Lie group actions can significantly reduce the sample complexity for estimating various probability divergences and density estimation, providing new bounds especially for groups of positive dimension.

Contribution

It introduces novel bounds on sample complexity improvements due to invariances under Lie group actions, extending previous results for finite groups to continuous groups.

Findings

Sample complexity is reduced proportionally to group size or quotient volume.

Convergence rate exponents are improved for positive-dimensional groups.

New bounds are established for groups of positive dimension.

Abstract

Group-invariant probability distributions appear in many data-generative models in machine learning, such as graphs, point clouds, and images. In practice, one often needs to estimate divergences between such distributions. In this work, we study how the inherent invariances, with respect to any smooth action of a Lie group on a manifold, improve sample complexity when estimating the 1-Wasserstein distance, the Sobolev Integral Probability Metrics (Sobolev IPMs), the Maximum Mean Discrepancy (MMD), and also the complexity of the density estimation problem (in the $L^2$ and $L^\infty$ distance). Our results indicate a two-fold gain: (1) reducing the sample complexity by a multiplicative factor corresponding to the group size (for finite groups) or the normalized volume of the quotient space (for groups of positive dimension); (2) improving the exponent in the convergence rate (for groups of positive dimension). These results are completely new for groups of positive dimension and extend recent bounds for finite group actions.