Neural Isometries: Taming Transformations for Equivariant ML

TL;DR

Neural Isometries introduces an autoencoder framework that learns to encode geometric observations into a latent space where isometries are preserved, enabling effective self-supervised learning and pose regression in complex symmetry scenarios.

Contribution

The paper proposes Neural Isometries, a novel autoencoder approach that learns to encode observations into a space where geometric symmetries are preserved, facilitating equivariant learning and pose estimation.

Findings

Achieves competitive results with handcrafted symmetry-aware networks

Enables direct camera pose regression from learned encodings

Provides a flexible framework for handling complex nonlinear symmetries

Abstract

Real-world geometry and 3D vision tasks are replete with challenging symmetries that defy tractable analytical expression. In this paper, we introduce Neural Isometries, an autoencoder framework which learns to map the observation space to a general-purpose latent space wherein encodings are related by isometries whenever their corresponding observations are geometrically related in world space. Specifically, we regularize the latent space such that maps between encodings preserve a learned inner product and commute with a learned functional operator, in the same manner as rigid-body transformations commute with the Laplacian. This approach forms an effective backbone for self-supervised representation learning, and we demonstrate that a simple off-the-shelf equivariant network operating in the pre-trained latent space can achieve results on par with meticulously-engineered, handcrafted networks designed to handle complex, nonlinear symmetries. Furthermore, isometric maps capture information about the respective transformations in world space, and we show that this allows us to regress camera poses directly from the coefficients of the maps between encodings of adjacent views of a scene.