Algebraic Priors for Approximately Equivariant Networks

TL;DR

This paper introduces a parameter-free, algebraic approach to equivariant neural networks using group representation theory, achieving competitive performance without architecture-specific constraints.

Contribution

It proves that the latent space must contain the regular representation for equivariance and leverages this insight as a parameter-free inductive bias in neural networks.

Findings

The method matches or outperforms specialized models in various tasks.

The regular representation as a bias outperforms trivial and defining representations.

The approach works well even for infinite groups.

Abstract

Equivariant neural networks incorporate symmetries through group actions, embedding them as an inductive bias to improve performance. Existing methods learn an equivariant action on the latent space, or design architectures that are equivariant by construction. These approaches often deliver strong empirical results but can involve architecture-specific constraints, large parameter counts, and high computational cost. We challenge the paradigm of complex equivariant architectures with a parameter-free approach grounded in group representation theory. We prove that for an equivariant encoder over a finite group, the latent space must almost surely contain one copy of its regular representation for each linearly independent data orbit, which we explore with a number of empirical studies. Leveraging this foundational algebraic insight, we impose the group's regular representation as an inductive bias via an auxiliary loss, adding no learnable parameters. Our extensive evaluation shows that this method matches or outperforms specialized models in several cases, even those for infinite groups. We further validate our choice of the regular representation through an ablation study, showing it consistently outperforms defining and trivial group representation baselines.