The High Cost of Data Augmentation for Learning Equivariant Models

TL;DR

This paper compares two data augmentation methods for learning models with continuous symmetries, showing that quadrature-based augmentation preserves symmetry exactly in polynomial models, unlike random sampling which converges more slowly.

Contribution

It introduces and analyzes quadrature and random sampling approaches for symmetry augmentation, highlighting the superior symmetry preservation of quadrature methods.

Findings

Quadrature augmentation achieves exact symmetry in polynomial models.

Random sampling of Haar measure converges with square-root rate.

Quadrature method outperforms random sampling in symmetry preservation.

Abstract

According to Noether's theorem the presence of a continuous symmetry in a Hamiltonian systems is equivalent to the existence of a conserved quantity, yet these symmetries are not always explicitly enforced in data-driven models. There remains a debate whether or not encoding of symmetry into a model architecture is the optimal approach. A competing approach is to target approximate symmetry through data augmentation. In this work, we study two approaches aimed at improving the symmetry properties of such an approximation scheme: one based on a quadrature rule for the Haar measure on the compact Lie group encoding the continuous symmetry of interest and one based on a random sampling of that Haar measure. We demonstrate both theoretically and empirically that the quadrature augmentation leads to exact symmetry preservation in polynomial models, while the random augmentation has only square-root convergence of the symmetrization error.