Learning with 3D rotations, a hitchhiker's guide to SO(3)

TL;DR

This paper surveys various rotation representations in machine learning, analyzing their properties and providing guidance on selecting suitable representations for different scenarios involving 3D rotations.

Contribution

It offers a comprehensive overview of rotation representations, highlighting their properties and providing practical guidance for deep learning applications involving SO(3).

Findings

Certain representations facilitate gradient-based optimization

Guidelines depend on data characteristics and model input/output

Consolidates insights for better rotation representation choice

Abstract

Many settings in machine learning require the selection of a rotation representation. However, choosing a suitable representation from the many available options is challenging. This paper acts as a survey and guide through rotation representations. We walk through their properties that harm or benefit deep learning with gradient-based optimization. By consolidating insights from rotation-based learning, we provide a comprehensive overview of learning functions with rotation representations. We provide guidance on selecting representations based on whether rotations are in the model's input or output and whether the data primarily comprises small angles.