On Universality Classes of Equivariant Networks

TL;DR

This paper explores the approximation capabilities of equivariant neural networks, revealing that separation power alone does not determine universality and identifying conditions under which these models can or cannot approximate functions.

Contribution

It characterizes the universality classes of shallow invariant networks and provides conditions affecting the universality of equivariant models based on group structure.

Findings

Separation power does not fully determine approximation ability.

Conditions for shallow equivariant networks to be universal are identified.

Structural properties of symmetry groups influence model universality.

Abstract

Equivariant neural networks provide a principled framework for incorporating symmetry into learning architectures and have been extensively analyzed through the lens of their separation power, that is, the ability to distinguish inputs modulo symmetry. This notion plays a central role in settings such as graph learning, where it is often formalized via the Weisfeiler-Leman hierarchy. In contrast, the universality of equivariant models-their capacity to approximate target functions-remains comparatively underexplored. In this work, we investigate the approximation power of equivariant neural networks beyond separation constraints. We show that separation power does not fully capture expressivity: models with identical separation power may differ in their approximation ability. To demonstrate this, we characterize the universality classes of shallow invariant networks, providing a general framework for understanding which functions these architectures can approximate. Since equivariant models reduce to invariant ones under projection, this analysis yields sufficient conditions under which shallow equivariant networks fail to be universal. Conversely, we identify settings where shallow models do achieve separation-constrained universality. These positive results, however, depend critically on structural properties of the symmetry group, such as the existence of adequate normal subgroups, which may not hold in important cases like permutation symmetry.