How unconstrained machine-learning models learn physical symmetries

TL;DR

This paper investigates how unconstrained machine-learning models learn physical symmetries, introduces metrics to measure symmetry learning, and demonstrates that minimal inductive biases can improve model stability and physical fidelity.

Contribution

It provides a rigorous framework for diagnosing symmetry learning in unconstrained models and shows how minimal biases enhance physical accuracy without sacrificing expressivity.

Findings

Unconstrained models can learn approximate symmetries with data augmentation.

Metrics reveal how symmetry information propagates across layers.

Minimal inductive biases improve model stability and physical fidelity.

Abstract

The requirement of generating predictions that exactly fulfill the fundamental symmetry of the corresponding physical quantities has profoundly shaped the development of machine-learning models for physical simulations. In many cases, models are built using constrained mathematical forms that ensure that symmetries are enforced exactly. However, unconstrained models that do not obey rotational symmetries are often found to have competitive performance, and to be able to \emph{learn} to a high level of accuracy an approximate equivariant behavior with a simple data augmentation strategy. In this paper, we introduce rigorous metrics to measure the symmetry content of the learned representations in such models, and assess the accuracy by which the outputs fulfill the equivariant condition. We apply these metrics to two unconstrained, transformer-based models operating on decorated point clouds (a graph neural network for atomistic simulations and a PointNet-style architecture for particle physics) to investigate how symmetry information is processed across architectural layers and is learned during training. Based on these insights, we establish a rigorous framework for diagnosing spectral failure modes in ML models. Enabled by this analysis, we demonstrate that one can achieve superior stability and accuracy by strategically injecting the minimum required inductive biases, preserving the high expressivity and scalability of unconstrained architectures while guaranteeing physical fidelity.