Steerable Transformers for Volumetric Data

TL;DR

This paper presents Steerable Transformers, an extension of Vision Transformers that maintains SE(d) equivariance, improving performance on volumetric data by integrating steerable convolutions and Fourier space nonlinearities.

Contribution

It introduces an SE(d)-equivariant attention mechanism using steerable convolutions and Fourier space nonlinearities, enhancing volumetric data processing.

Findings

Enhanced performance on 2D and 3D datasets.

SE(d) equivariance improves model robustness.

Fourier space nonlinearities contribute to better feature learning.

Abstract

We introduce Steerable Transformers, an extension of the Vision Transformer mechanism that maintains equivariance to the special Euclidean group $\mathrm{SE}(d)$. We propose an equivariant attention mechanism that operates on features extracted by steerable convolutions. Operating in Fourier space, our network utilizes Fourier space non-linearities. Our experiments in both two and three dimensions show that adding steerable transformer layers to steerable convolutional networks enhances performance.