Symmetry From Scratch: Group Equivariance as a Supervised Learning Task

TL;DR

This paper introduces symmetry-cloning, a supervised learning method that enables general neural networks to learn and incorporate symmetries directly, reducing the need for specialized equivariant architectures.

Contribution

It proposes a novel approach allowing standard models to learn symmetries from group-equivariant architectures, simplifying symmetry integration in machine learning.

Findings

Models can learn symmetries directly from data.

Symmetry learning improves model flexibility and generalization.

Method retains or breaks symmetries for different tasks.

Abstract

In machine learning datasets with symmetries, the paradigm for backward compatibility with symmetry-breaking has been to relax equivariant architectural constraints, engineering extra weights to differentiate symmetries of interest. However, this process becomes increasingly over-engineered as models are geared towards specific symmetries/asymmetries hardwired of a particular set of equivariant basis functions. In this work, we introduce symmetry-cloning, a method for inducing equivariance in machine learning models. We show that general machine learning architectures (i.e., MLPs) can learn symmetries directly as a supervised learning task from group equivariant architectures and retain/break the learned symmetry for downstream tasks. This simple formulation enables machine learning models with group-agnostic architectures to capture the inductive bias of group-equivariant architectures.