Symmetry Breaking and Equivariant Neural Networks

TL;DR

This paper explores the limitations of equivariant neural networks in breaking symmetry at the data sample level and introduces a relaxed equivariance concept to address this, with applications across multiple domains.

Contribution

It introduces the concept of relaxed equivariance to overcome symmetry-breaking limitations in neural networks and integrates it into equivariant multilayer perceptrons.

Findings

Relaxed equivariance enables symmetry breaking in neural networks.

Incorporating relaxed equivariance improves model flexibility.

Applications span physics, graph learning, and optimization.

Abstract

Using symmetry as an inductive bias in deep learning has been proven to be a principled approach for sample-efficient model design. However, the relationship between symmetry and the imperative for equivariance in neural networks is not always obvious. Here, we analyze a key limitation that arises in equivariant functions: their incapacity to break symmetry at the level of individual data samples. In response, we introduce a novel notion of 'relaxed equivariance' that circumvents this limitation. We further demonstrate how to incorporate this relaxation into equivariant multilayer perceptrons (E-MLPs), offering an alternative to the noise-injection method. The relevance of symmetry breaking is then discussed in various application domains: physics, graph representation learning, combinatorial optimization and equivariant decoding.