Sample Complexity of Probability Divergences under Group Symmetry

TL;DR

This paper analyzes how group symmetry can reduce the sample complexity in estimating probability divergences, showing proportional improvements for finite groups and more nuanced effects for infinite groups and MMD.

Contribution

It extends previous work by providing a rigorous analysis of sample complexity improvements under both finite and infinite group symmetries, including asymmetric divergences and MMD.

Findings

Sample complexity reduction is proportional to group size for finite groups.

For infinite groups like compact Lie groups, the rate depends on the intrinsic dimension.

Numerical simulations confirm the theoretical predictions.

Abstract

We rigorously quantify the improvement in the sample complexity of variational divergence estimations for group-invariant distributions. In the cases of the Wasserstein-1 metric and the Lipschitz-regularized $\alpha$-divergences, the reduction of sample complexity is proportional to the group size if the group is finite. In addition to the published version at ICML 2023, our proof indeed has included the case when the group is infinite such as compact Lie groups, the convergence rate can be further improved and depends on the intrinsic dimension of the fundamental domain characterized by the scaling of its covering number. Our approach is different from that in [Tahmasebi & Jegelka, ICML 2024] and our work also applies to asymmetric divergences, such as the Lipschitz-regularized $\alpha$-divergences. For the maximum mean discrepancy (MMD), the improvement of sample complexity is more nuanced, as it depends on not only the group size but also the choice of kernel. Numerical simulations verify our theories.