Scalable Scene Flow from Point Clouds in the Real World

TL;DR

This paper introduces a large-scale real-world dataset for scene flow estimation from point clouds, along with a new architecture that enables real-time inference, addressing data limitations and performance degradation issues.

Contribution

The paper presents a significantly larger dataset for scene flow, and a novel FastFlow3D model that achieves real-time performance on full point clouds.

Findings

Larger dataset improves scene flow prediction accuracy.

Down-sampling degrades performance, motivating full point cloud models.

FastFlow3D achieves real-time inference on full point clouds.

Abstract

Autonomous vehicles operate in highly dynamic environments necessitating an accurate assessment of which aspects of a scene are moving and where they are moving to. A popular approach to 3D motion estimation, termed scene flow, is to employ 3D point cloud data from consecutive LiDAR scans, although such approaches have been limited by the small size of real-world, annotated LiDAR data. In this work, we introduce a new large-scale dataset for scene flow estimation derived from corresponding tracked 3D objects, which is $\sim$1,000$\times$ larger than previous real-world datasets in terms of the number of annotated frames. We demonstrate how previous works were bounded based on the amount of real LiDAR data available, suggesting that larger datasets are required to achieve state-of-the-art predictive performance. Furthermore, we show how previous heuristics for operating on point clouds such as down-sampling heavily degrade performance, motivating a new class of models that are tractable on the full point cloud. To address this issue, we introduce the FastFlow3D architecture which provides real time inference on the full point cloud. Additionally, we design human-interpretable metrics that better capture real world aspects by accounting for ego-motion and providing breakdowns per object type. We hope that this dataset may provide new opportunities for developing real world scene flow systems.