Motion Cues from Image-based Point Tracking for LiDAR Scene Flow Estimation

TL;DR

This paper introduces TrackCue, a novel framework that leverages dense image-space trajectories and motion compensation to improve static-dynamic classification and scene flow estimation in LiDAR data.

Contribution

TrackCue repurposes point tracking and visual motion cues to enhance dynamic object representation and label refinement in LiDAR scene flow estimation.

Findings

Significantly improves static-dynamic classification accuracy.

Enhances self-supervised scene flow estimation performance.

Provides more reliable supervision through refined motion cues.

Abstract

LiDAR scene flow estimation is essential for autonomous driving, as it provides 3D motion for each point. Self-supervised approaches use static-dynamic classification to mitigate the imbalance between static and dynamic points, deriving targeted supervision. However, existing methods rely on sparse geometric observations for this classification, making them vulnerable to data sparsity and occlusions. The resulting noisy labels provide incorrect motion guidance and degrade scene flow learning. To address this, we introduce TrackCue, a tracking-guided framework for improving dynamic object representation in LiDAR scene flow estimation. In particular, TrackCue repurposes point tracking to obtain dense image-space trajectories anchored to LiDAR points, providing motion cues beyond sparse geometric observations. Furthermore, we present a visually consistent motion compensation strategy that compares the tracked trajectories with ego-induced rigid trajectories in the image plane, effectively isolating true object motion from ego-induced apparent motion. To transfer these isolated motion cues back to the LiDAR domain, we perform visual motion cue lifting, which associates ego-compensated image trajectories with LiDAR points for static-dynamic label refinement. As a result, TrackCue produces more accurate static-dynamic classification and provides more reliable supervision for scene flow learning. Experimental results show that TrackCue significantly improves the precision and F1 score of dynamic labels, leading to performance gains in self-supervised scene flow estimation.