Sparse Convolutions on Continuous Domains for Point Cloud and Event Stream Networks

TL;DR

This paper introduces a new sparse convolution operator for continuous, unstructured data like point clouds and event streams, enabling faster training and state-of-the-art performance.

Contribution

It presents a mathematically consistent, efficient sparse convolution method for continuous data, advancing deep learning capabilities in 3D and event-based vision.

Findings

Faster training speeds on point cloud classification

Comparable accuracy with significantly less memory

State-of-the-art results on event stream tasks

Abstract

Image convolutions have been a cornerstone of a great number of deep learning advances in computer vision. The research community is yet to settle on an equivalent operator for sparse, unstructured continuous data like point clouds and event streams however. We present an elegant sparse matrix-based interpretation of the convolution operator for these cases, which is consistent with the mathematical definition of convolution and efficient during training. On benchmark point cloud classification problems we demonstrate networks built with these operations can train an order of magnitude or more faster than top existing methods, whilst maintaining comparable accuracy and requiring a tiny fraction of the memory. We also apply our operator to event stream processing, achieving state-of-the-art results on multiple tasks with streams of hundreds of thousands of events.