RAFT-MSF: Self-Supervised Monocular Scene Flow using Recurrent Optimizer

TL;DR

This paper presents RAFT-MSF, a self-supervised monocular scene flow method that significantly improves accuracy over previous approaches by iteratively updating 3D motion and disparity, achieving state-of-the-art results.

Contribution

It introduces a novel decoder based on RAFT for joint 3D motion and disparity estimation, with enhancements that boost accuracy and runtime efficiency.

Findings

Achieves up to 7.2% accuracy improvement over previous methods.

Outperforms semi-supervised methods in accuracy while being 228 times faster.

Sets new state-of-the-art among self-supervised monocular scene flow techniques.

Abstract

Learning scene flow from a monocular camera still remains a challenging task due to its ill-posedness as well as lack of annotated data. Self-supervised methods demonstrate learning scene flow estimation from unlabeled data, yet their accuracy lags behind (semi-)supervised methods. In this paper, we introduce a self-supervised monocular scene flow method that substantially improves the accuracy over the previous approaches. Based on RAFT, a state-of-the-art optical flow model, we design a new decoder to iteratively update 3D motion fields and disparity maps simultaneously. Furthermore, we propose an enhanced upsampling layer and a disparity initialization technique, which overall further improves accuracy up to 7.2%. Our method achieves state-of-the-art accuracy among all self-supervised monocular scene flow methods, improving accuracy by 34.2%. Our fine-tuned model outperforms the best previous semi-supervised method with 228 times faster runtime. Code will be publicly available.