Equivariance and generalization in neural networks

TL;DR

This paper demonstrates that neural networks incorporating translational equivariance outperform non-equivariant models in physics-related tasks, showing improved performance and generalization across parameters and lattice sizes.

Contribution

It systematically studies the impact of translational equivariance in neural networks applied to high energy physics, highlighting its benefits for performance and generalization.

Findings

Equivariant networks outperform non-equivariant ones in various tasks.

Equivariant models generalize better to unseen parameters.

Performance gains are consistent across different lattice sizes.

Abstract

The crucial role played by the underlying symmetries of high energy physics and lattice field theories calls for the implementation of such symmetries in the neural network architectures that are applied to the physical system under consideration. In these proceedings, we focus on the consequences of incorporating translational equivariance among the network properties, particularly in terms of performance and generalization. The benefits of equivariant networks are exemplified by studying a complex scalar field theory, on which various regression and classification tasks are examined. For a meaningful comparison, promising equivariant and non-equivariant architectures are identified by means of a systematic search. The results indicate that in most of the tasks our best equivariant architectures can perform and generalize significantly better than their non-equivariant counterparts, which applies not only to physical parameters beyond those represented in the training set, but also to different lattice sizes.