An Analysis of SVD for Deep Rotation Estimation

TL;DR

This paper investigates the use of SVD orthogonalization for representing 3D rotations in neural networks, demonstrating its theoretical suitability and superior performance over traditional methods in various deep learning tasks.

Contribution

It provides a theoretical justification for using SVD orthogonalization for rotations and shows it outperforms existing representations in deep learning applications.

Findings

SVD orthogonalization is theoretically optimal for rotation projection.

Replacing traditional rotation representations with SVD improves performance.

SVD-based methods achieve state-of-the-art results in multiple applications.

Abstract

Symmetric orthogonalization via SVD, and closely related procedures, are well-known techniques for projecting matrices onto $O(n)$ or $SO(n)$. These tools have long been used for applications in computer vision, for example optimal 3D alignment problems solved by orthogonal Procrustes, rotation averaging, or Essential matrix decomposition. Despite its utility in different settings, SVD orthogonalization as a procedure for producing rotation matrices is typically overlooked in deep learning models, where the preferences tend toward classic representations like unit quaternions, Euler angles, and axis-angle, or more recently-introduced methods. Despite the importance of 3D rotations in computer vision and robotics, a single universally effective representation is still missing. Here, we explore the viability of SVD orthogonalization for 3D rotations in neural networks. We present a theoretical analysis that shows SVD is the natural choice for projecting onto the rotation group. Our extensive quantitative analysis shows simply replacing existing representations with the SVD orthogonalization procedure obtains state of the art performance in many deep learning applications covering both supervised and unsupervised training.