Learning from Frustration: Torsor CNNs on Graphs

TL;DR

Torsor CNNs introduce a novel framework for learning on graphs with local symmetries, enabling equivariance to local transformations and unifying various architectures for applications like multi-view 3D recognition.

Contribution

The paper develops Torsor CNNs, a new geometric framework that handles local symmetries on graphs, generalizing existing models and connecting to group synchronization.

Findings

Torsor CNNs are provably equivariant to local coordinate changes.

The frustration loss regularizes local equivariance in neural networks.

Application to multi-view 3D recognition demonstrates practical effectiveness.

Abstract

Most equivariant neural networks rely on a single global symmetry, limiting their use in domains where symmetries are instead local. We introduce Torsor CNNs, a framework for learning on graphs with local symmetries encoded as edge potentials -- group-valued transformations between neighboring coordinate frames. We establish that this geometric construction is fundamentally equivalent to the classical group synchronization problem, yielding: (1) a Torsor Convolutional Layer that is provably equivariant to local changes in coordinate frames, and (2) the frustration loss -- a standalone geometric regularizer that encourages locally equivariant representations when added to any NN's training objective. The Torsor CNN framework unifies and generalizes several architectures -- including classical CNNs and Gauge CNNs on manifolds -- by operating on arbitrary graphs without requiring a global coordinate system or smooth manifold structure. We establish the mathematical foundations of this framework and demonstrate its applicability to multi-view 3D recognition, where relative camera poses naturally define the required edge potentials.