PointPWC-Net: A Coarse-to-Fine Network for Supervised and Self-Supervised Scene Flow Estimation on 3D Point Clouds

TL;DR

PointPWC-Net is an innovative deep learning model that estimates scene flow from 3D point clouds using a coarse-to-fine approach, novel cost volume handling, and self-supervised training, achieving state-of-the-art results and strong generalization.

Contribution

The paper introduces a new end-to-end scene flow network with a point-based cost volume and self-supervised learning, advancing 3D scene flow estimation methods.

Findings

Outperforms state-of-the-art on FlyingThings3D dataset.

Achieves comparable results with self-supervised training.

Demonstrates strong generalization on KITTI dataset.

Abstract

We propose a novel end-to-end deep scene flow model, called PointPWC-Net, on 3D point clouds in a coarse-to-fine fashion. Flow computed at the coarse level is upsampled and warped to a finer level, enabling the algorithm to accommodate for large motion without a prohibitive search space. We introduce novel cost volume, upsampling, and warping layers to efficiently handle 3D point cloud data. Unlike traditional cost volumes that require exhaustively computing all the cost values on a high-dimensional grid, our point-based formulation discretizes the cost volume onto input 3D points, and a PointConv operation efficiently computes convolutions on the cost volume. Experiment results on FlyingThings3D outperform the state-of-the-art by a large margin. We further explore novel self-supervised losses to train our model and achieve comparable results to state-of-the-art trained with supervised loss. Without any fine-tuning, our method also shows great generalization ability on KITTI Scene Flow 2015 dataset, outperforming all previous methods.