Flopping for FLOPs: Leveraging equivariance for computational efficiency

TL;DR

This paper presents equivariant neural networks that leverage horizontal mirroring symmetry to reduce FLOPs and improve computational efficiency without sacrificing invariance, enabling scalable symmetry-aware models.

Contribution

It introduces a novel parametrization of feature spaces using mirror-symmetric and antisymmetric components, leading to block-diagonal linear layers that halve FLOPs compared to standard networks.

Findings

Reduces FLOPs by approximately 50% in equivariant networks.

Maintains symmetry invariance while improving computational efficiency.

Achieves faster wall-clock times for symmetry-aware architectures.

Abstract

Incorporating geometric invariance into neural networks enhances parameter efficiency but typically increases computational costs. This paper introduces new equivariant neural networks that preserve symmetry while maintaining a comparable number of floating-point operations (FLOPs) per parameter to standard non-equivariant networks. We focus on horizontal mirroring (flopping) invariance, common in many computer vision tasks. The main idea is to parametrize the feature spaces in terms of mirror-symmetric and mirror-antisymmetric features, i.e., irreps of the flopping group. This decomposes the linear layers to be block-diagonal, requiring half the number of FLOPs. Our approach reduces both FLOPs and wall-clock time, providing a practical solution for efficient, scalable symmetry-aware architectures.