Learning Linear Symmetries in Data Using Moment Matching

TL;DR

This paper introduces a method to learn symmetries in data distributions directly from data using eigenvector analysis of covariance matrices, addressing the problem's complexity and proposing solutions for orthogonal symmetries.

Contribution

It provides a theoretical framework and practical algorithms for identifying symmetries in data based on eigenvector properties of covariance matrices, especially for orthogonal symmetries.

Findings

Eigenvector-based methods effectively identify symmetries when covariance matrices have unique eigenvalues.

The problem simplifies to eigenvalue sign determination for orthogonal symmetries.

The proposed methods outperform baseline approaches in empirical tests.

Abstract

It is common in machine learning and statistics to use symmetries derived from expert knowledge to simplify problems or improve performance, using methods like data augmentation or penalties. In this paper we consider the unsupervised and semi-supervised problems of learning such symmetries in a distribution directly from data in a model-free fashion. We show that in the worst case this problem is as difficult as the graph automorphism problem. However, if we restrict to the case where the covariance matrix has unique eigenvalues, then the eigenvectors will also be eigenvectors of the symmetry transformation. If we further restrict to finding orthogonal symmetries, then the eigenvalues will be either be 1 or -1, and the problem reduces to determining which eigenvectors are which. We develop and compare theoretically and empirically the effectiveness of different methods of selecting which eigenvectors should have eigenvalue -1 in the symmetry transformation, and discuss how to extend this approach to non-orthogonal cases where we have labels