Probing the effects of broken symmetries in machine learning

TL;DR

This paper investigates the impact of approximate rotational symmetry in machine learning models for physical systems, finding minimal effects in many scenarios and discussing methods to mitigate symmetry breaking.

Contribution

It demonstrates that models with approximate rotational invariance perform well in physical predictions, challenging the necessity of strict symmetry enforcement in all cases.

Findings

Negligible effects of symmetry breaking in interpolative regimes

Model stability in gas-phase extrapolations despite symmetry artifacts

Strategies to reduce symmetry breaking improve observable convergence

Abstract

Symmetry is one of the most central concepts in physics, and it is no surprise that it has also been widely adopted as an inductive bias for machine-learning models applied to the physical sciences. This is especially true for models targeting the properties of matter at the atomic scale. Both established and state-of-the-art approaches, with almost no exceptions, are built to be exactly equivariant to translations, permutations, and rotations of the atoms. Incorporating symmetries -- rotations in particular -- constrains the model design space and implies more complicated architectures that are often also computationally demanding. There are indications that non-symmetric models can easily learn symmetries from data, and that doing so can even be beneficial for the accuracy of the model. We put a model that obeys rotational invariance only approximately to the test, in realistic scenarios involving simulations of gas-phase, liquid, and solid water. We focus specifically on physical observables that are likely to be affected -- directly or indirectly -- by symmetry breaking, finding negligible consequences when the model is used in an interpolative, bulk, regime. Even for extrapolative gas-phase predictions, the model remains very stable, even though symmetry artifacts are noticeable. We also discuss strategies that can be used to systematically reduce the magnitude of symmetry breaking when it occurs, and assess their impact on the convergence of observables.