MonoComb: A Sparse-to-Dense Combination Approach for Monocular Scene Flow

TL;DR

MonoComb introduces a novel monocular scene flow method that combines optical flow and depth estimation to produce dense scene flow, outperforming existing monocular approaches on KITTI 2015.

Contribution

It proposes a sparse-to-dense combination approach leveraging recent advances in depth and optical flow estimation for monocular scene flow.

Findings

Outperforms existing monocular methods in dynamic regions

Achieves second-best results on KITTI 2015 scene flow benchmark

Effectively interpolates occluded areas using optical flow and depth

Abstract

Contrary to the ongoing trend in automotive applications towards usage of more diverse and more sensors, this work tries to solve the complex scene flow problem under a monocular camera setup, i.e. using a single sensor. Towards this end, we exploit the latest achievements in single image depth estimation, optical flow, and sparse-to-dense interpolation and propose a monocular combination approach (MonoComb) to compute dense scene flow. MonoComb uses optical flow to relate reconstructed 3D positions over time and interpolates occluded areas. This way, existing monocular methods are outperformed in dynamic foreground regions which leads to the second best result among the competitors on the challenging KITTI 2015 scene flow benchmark.