Exploiting Semantic Information and Deep Matching for Optical Flow

TL;DR

This paper presents a novel optical flow estimation method for autonomous driving that leverages semantic segmentation, epipolar constraints, and a deep matching network with uncertainty estimation, achieving state-of-the-art results on KITTI 2015.

Contribution

It introduces a combined approach using semantic segmentation, epipolar constraints, and a new deep matching network with uncertainty estimation for improved optical flow accuracy.

Findings

Outperforms existing methods on KITTI 2015 benchmark

Effectively estimates traffic participants and static background

Utilizes epipolar constraints for faster, more accurate motion estimation

Abstract

We tackle the problem of estimating optical flow from a monocular camera in the context of autonomous driving. We build on the observation that the scene is typically composed of a static background, as well as a relatively small number of traffic participants which move rigidly in 3D. We propose to estimate the traffic participants using instance-level segmentation. For each traffic participant, we use the epipolar constraints that govern each independent motion for faster and more accurate estimation. Our second contribution is a new convolutional net that learns to perform flow matching, and is able to estimate the uncertainty of its matches. This is a core element of our flow estimation pipeline. We demonstrate the effectiveness of our approach in the challenging KITTI 2015 flow benchmark, and show that our approach outperforms published approaches by a large margin.