# Mono-SF: Multi-View Geometry Meets Single-View Depth for Monocular Scene Flow Estimation of Dynamic Traffic Scenes

## TL;DR

Mono-SF introduces a novel monocular scene flow estimation method that combines multi-view geometry with single-view depth distributions, enabling accurate 3D scene understanding from a single image in dynamic traffic scenes.

## Contribution

The paper presents Mono-SF, a new approach that jointly estimates 3D structure and motion from monocular images by integrating multi-view geometry with probabilistic depth estimation using ProbDepthNet.

## Key findings

- Mono-SF outperforms existing monocular scene flow methods.
- ProbDepthNet effectively estimates pixel-wise depth distributions.
- The approach demonstrates robustness in dynamic traffic scene scenarios.

## Abstract

Existing 3D scene flow estimation methods provide the 3D geometry and 3D motion of a scene and gain a lot of interest, for example in the context of autonomous driving. These methods are traditionally based on a temporal series of stereo images. In this paper, we propose a novel monocular 3D scene flow estimation method, called Mono-SF. Mono-SF jointly estimates the 3D structure and motion of the scene by combining multi-view geometry and single-view depth information. Mono-SF considers that the scene flow should be consistent in terms of warping the reference image in the consecutive image based on the principles of multi-view geometry. For integrating single-view depth in a statistical manner, a convolutional neural network, called ProbDepthNet, is proposed. ProbDepthNet estimates pixel-wise depth distributions from a single image rather than single depth values. Additionally, as part of ProbDepthNet, a novel recalibration technique for regression problems is proposed to ensure well-calibrated distributions. Our experiments show that Mono-SF outperforms state-of-the-art monocular baselines and ablation studies support the Mono-SF approach and ProbDepthNet design.