Just Go with the Flow: Self-Supervised Scene Flow Estimation

TL;DR

This paper introduces a self-supervised approach for scene flow estimation in autonomous driving, enabling training on unlabeled data and achieving state-of-the-art results without relying on manual annotations.

Contribution

The authors propose a novel self-supervised training method for scene flow that uses nearest neighbors and cycle consistency losses, eliminating the need for labeled data.

Findings

Matches supervised state-of-the-art performance without annotations

Outperforms supervised methods when combined with small labeled datasets

Enables training on large unlabeled autonomous driving datasets

Abstract

When interacting with highly dynamic environments, scene flow allows autonomous systems to reason about the non-rigid motion of multiple independent objects. This is of particular interest in the field of autonomous driving, in which many cars, people, bicycles, and other objects need to be accurately tracked. Current state-of-the-art methods require annotated scene flow data from autonomous driving scenes to train scene flow networks with supervised learning. As an alternative, we present a method of training scene flow that uses two self-supervised losses, based on nearest neighbors and cycle consistency. These self-supervised losses allow us to train our method on large unlabeled autonomous driving datasets; the resulting method matches current state-of-the-art supervised performance using no real world annotations and exceeds state-of-the-art performance when combining our self-supervised approach with supervised learning on a smaller labeled dataset.