Rotation-Invariant Random Features Provide a Strong Baseline for Machine Learning on 3D Point Clouds

TL;DR

This paper introduces a simple, fast, rotation-invariant random features method for 3D point cloud data, serving as a strong baseline that matches or exceeds neural network performance on molecular and shape classification tasks.

Contribution

It extends Rahimi & Recht's random features to be rotation-invariant for 3D data, providing a general-purpose, efficient baseline for machine learning on point clouds.

Findings

Outperforms or matches neural networks on QM7 and QM9 datasets.

Provides a fast, rotation-invariant baseline for ModelNet40 shape classification.

Achieves significantly lower prediction latency than kernel methods.

Abstract

Rotational invariance is a popular inductive bias used by many fields in machine learning, such as computer vision and machine learning for quantum chemistry. Rotation-invariant machine learning methods set the state of the art for many tasks, including molecular property prediction and 3D shape classification. These methods generally either rely on task-specific rotation-invariant features, or they use general-purpose deep neural networks which are complicated to design and train. However, it is unclear whether the success of these methods is primarily due to the rotation invariance or the deep neural networks. To address this question, we suggest a simple and general-purpose method for learning rotation-invariant functions of three-dimensional point cloud data using a random features approach. Specifically, we extend the random features method of Rahimi & Recht 2007 by deriving a version that is invariant to three-dimensional rotations and showing that it is fast to evaluate on point cloud data. We show through experiments that our method matches or outperforms the performance of general-purpose rotation-invariant neural networks on standard molecular property prediction benchmark datasets QM7 and QM9. We also show that our method is general-purpose and provides a rotation-invariant baseline on the ModelNet40 shape classification task. Finally, we show that our method has an order of magnitude smaller prediction latency than competing kernel methods.