Tunable Soft Equivariance with Guarantees

TL;DR

This paper introduces a framework for creating soft equivariant models that allow adjustable equivariance levels, improving performance and reducing error across various vision tasks and architectures.

Contribution

It presents a general method to control equivariance in pre-trained models through weight projection, with theoretical guarantees and broad applicability.

Findings

Improves performance on ImageNet benchmark.

Reduces equivariance error in multiple tasks.

Applicable to architectures like ViT and ResNet.

Abstract

Equivariance is a fundamental property in computer vision models, yet strict equivariance is rarely satisfied in real-world data, which can limit a model's performance. Controlling the degree of equivariance is therefore desirable. We propose a general framework for constructing soft equivariant models by projecting the model weights into a designed subspace. The method applies to any pre-trained architecture and provides theoretical bounds on the induced equivariance error. Empirically, we demonstrate the effectiveness of our method on multiple pre-trained backbones, including ViT and ResNet, across image classification, semantic segmentation, and human-trajectory prediction tasks. Notably, our approach improves the performance while simultaneously reducing equivariance error on the competitive ImageNet benchmark.