Encoding Involutory Invariances in Neural Networks

TL;DR

This paper introduces neural network architectures that embed involutory invariance, ensuring predictions respect specific symmetries, with theoretical guarantees and improved performance on symmetry-invariant tasks.

Contribution

The work develops new neural network architectures that embed involutory invariance, providing rigorous proofs and demonstrating superior performance on symmetry-aware datasets.

Findings

The proposed models guarantee involutory invariance in predictions.

Experiments show improved accuracy over baseline models.

Models outperform physics-informed neural networks on symmetry tasks.

Abstract

In certain situations, neural networks are trained upon data that obey underlying symmetries. However, the predictions do not respect the symmetries exactly unless embedded in the network structure. In this work, we introduce architectures that embed a special kind of symmetry namely, invariance with respect to involutory linear/affine transformations up to parity $p=\pm 1$. We provide rigorous theorems to show that the proposed network ensures such an invariance and present qualitative arguments for a special universal approximation theorem. An adaption of our techniques to CNN tasks for datasets with inherent horizontal/vertical reflection symmetry is demonstrated. Extensive experiments indicate that the proposed model outperforms baseline feed-forward and physics-informed neural networks while identically respecting the underlying symmetry.