Occlusions, Motion and Depth Boundaries with a Generic Network for Disparity, Optical Flow or Scene Flow Estimation

TL;DR

This paper introduces a learning-based method that jointly estimates occlusions with disparity or optical flow, improving accuracy in boundary detection and enhancing performance in scene flow and motion segmentation tasks.

Contribution

A novel generic network architecture that jointly estimates occlusions and motion, achieving state-of-the-art results on benchmarks and improving downstream tasks.

Findings

Outperforms state-of-the-art in occlusion and boundary estimation

Achieves top performance on KITTI benchmark

Enhances motion segmentation and scene flow accuracy

Abstract

Occlusions play an important role in disparity and optical flow estimation, since matching costs are not available in occluded areas and occlusions indicate depth or motion boundaries. Moreover, occlusions are relevant for motion segmentation and scene flow estimation. In this paper, we present an efficient learning-based approach to estimate occlusion areas jointly with disparities or optical flow. The estimated occlusions and motion boundaries clearly improve over the state-of-the-art. Moreover, we present networks with state-of-the-art performance on the popular KITTI benchmark and good generic performance. Making use of the estimated occlusions, we also show improved results on motion segmentation and scene flow estimation.