A Smooth Representation of Belief over SO(3) for Deep Rotation Learning with Uncertainty

TL;DR

This paper introduces a smooth, symmetric matrix representation of SO(3) that enhances deep rotation learning by improving accuracy, convergence, and uncertainty estimation, especially in out-of-distribution scenarios.

Contribution

The authors propose a novel symmetric matrix representation of SO(3) that is smooth and encodes a belief over quaternions, enabling more accurate and uncertainty-aware rotation regression in neural networks.

Findings

Superior accuracy over existing rotation representations on synthetic data.

Effective out-of-distribution rejection improves robustness in real-world scenarios.

Supports uncertainty estimation without additional stochastic methods.

Abstract

Accurate rotation estimation is at the heart of robot perception tasks such as visual odometry and object pose estimation. Deep neural networks have provided a new way to perform these tasks, and the choice of rotation representation is an important part of network design. In this work, we present a novel symmetric matrix representation of the 3D rotation group, SO(3), with two important properties that make it particularly suitable for learned models: (1) it satisfies a smoothness property that improves convergence and generalization when regressing large rotation targets, and (2) it encodes a symmetric Bingham belief over the space of unit quaternions, permitting the training of uncertainty-aware models. We empirically validate the benefits of our formulation by training deep neural rotation regressors on two data modalities. First, we use synthetic point-cloud data to show that our representation leads to superior predictive accuracy over existing representations for arbitrary rotation targets. Second, we use image data collected onboard ground and aerial vehicles to demonstrate that our representation is amenable to an effective out-of-distribution (OOD) rejection technique that significantly improves the robustness of rotation estimates to unseen environmental effects and corrupted input images, without requiring the use of an explicit likelihood loss, stochastic sampling, or an auxiliary classifier. This capability is key for safety-critical applications where detecting novel inputs can prevent catastrophic failure of learned models.