TeFlow: Enabling Multi-frame Supervision for Self-Supervised Feed-forward Scene Flow Estimation

TL;DR

TeFlow introduces a multi-frame supervision approach with temporal ensembling for self-supervised scene flow estimation, significantly improving accuracy and speed over previous methods.

Contribution

It presents a novel temporal ensembling strategy that leverages multi-frame cues to enhance self-supervised scene flow estimation in a feed-forward model.

Findings

Achieves up to 33% performance improvement on Argoverse 2 and nuScenes datasets.

Speeds up scene flow estimation by 150 times compared to optimization-based methods.

Establishes a new state-of-the-art for self-supervised feed-forward scene flow estimation.

Abstract

Self-supervised feed-forward methods for scene flow estimation offer real-time efficiency, but their supervision from two-frame point correspondences is unreliable and often breaks down under occlusions. Multi-frame supervision has the potential to provide more stable guidance by incorporating motion cues from past frames, yet naive extensions of two-frame objectives are ineffective because point correspondences vary abruptly across frames, producing inconsistent signals. In the paper, we present TeFlow, enabling multi-frame supervision for feed-forward models by mining temporally consistent supervision. TeFlow introduces a temporal ensembling strategy that forms reliable supervisory signals by aggregating the most temporally consistent motion cues from a candidate pool built across multiple frames. Extensive evaluations demonstrate that TeFlow establishes a new state-of-the-art for self-supervised feed-forward methods, achieving performance gains of up to 33\% on the challenging Argoverse 2 and nuScenes datasets. Our method performs on par with leading optimization-based methods, yet speeds up 150 times. The code is open-sourced at https://github.com/Kin-Zhang/TeFlow along with trained model weights.