Probing Equivariance and Symmetry Breaking in Convolutional Networks

TL;DR

This paper investigates the effects of explicit symmetry priors in convolutional networks, introducing a flexible architecture to compare equivariant and non-equivariant models across tasks, revealing when symmetry constraints help or hinder performance.

Contribution

The authors present Rapidash, a unified architecture enabling controlled comparison of equivariant and non-equivariant models, and provide empirical insights into when symmetry constraints improve model performance.

Findings

Equivariant models outperform less constrained models when aligned with task geometry.

Increasing capacity does not fully close performance gaps between models.

Symmetry breaking via geometric reference frames improves performance across tasks.

Abstract

In this work, we explore the trade-offs of explicit structural priors, particularly group equivariance. We address this through theoretical analysis and a comprehensive empirical study. To enable controlled and fair comparisons, we introduce \texttt{Rapidash}, a unified group convolutional architecture that allows for different variants of equivariant and non-equivariant models. Our results suggest that more constrained equivariant models outperform less constrained alternatives when aligned with the geometry of the task, and increasing representation capacity does not fully eliminate performance gaps. We see improved performance of models with equivariance and symmetry-breaking through tasks like segmentation, regression, and generation across diverse datasets. Explicit \textit{symmetry breaking} via geometric reference frames consistently improves performance, while \textit{breaking equivariance} through geometric input features can be helpful when aligned with task geometry. Our results provide task-specific performance trends that offer a more nuanced way for model selection.