Euclidean, Projective, Conformal: Choosing a Geometric Algebra for Equivariant Transformers

TL;DR

This paper introduces a flexible geometric algebra transformer framework that can be adapted to Euclidean, projective, and conformal algebras, enabling scalable and effective 3D data processing.

Contribution

It generalizes the GATr architecture to any geometric algebra, analyzing and comparing Euclidean, projective, and conformal versions for 3D data tasks.

Findings

Conformal algebra provides the most powerful and performant architecture.

Euclidean version is computationally cheap but less expressive.

Projective algebra is less expressive but more efficient.

Abstract

The Geometric Algebra Transformer (GATr) is a versatile architecture for geometric deep learning based on projective geometric algebra. We generalize this architecture into a blueprint that allows one to construct a scalable transformer architecture given any geometric (or Clifford) algebra. We study versions of this architecture for Euclidean, projective, and conformal algebras, all of which are suited to represent 3D data, and evaluate them in theory and practice. The simplest Euclidean architecture is computationally cheap, but has a smaller symmetry group and is not as sample-efficient, while the projective model is not sufficiently expressive. Both the conformal algebra and an improved version of the projective algebra define powerful, performant architectures.