MirrorFlow: Exploiting Symmetries in Joint Optical Flow and Occlusion Estimation

TL;DR

MirrorFlow introduces a symmetric approach to optical flow and occlusion estimation, leveraging symmetry properties like forward-backward consistency and occlusion-disocclusion symmetry to improve accuracy, especially in challenging datasets.

Contribution

The paper presents a novel symmetric model that jointly estimates optical flow and occlusions, exploiting symmetry properties to enhance performance over existing methods.

Findings

Achieves state-of-the-art results on KITTI dataset

Demonstrates the effectiveness of symmetry-based occlusion reasoning

Improves optical flow accuracy by integrating occlusion cues

Abstract

Optical flow estimation is one of the most studied problems in computer vision, yet recent benchmark datasets continue to reveal problem areas of today's approaches. Occlusions have remained one of the key challenges. In this paper, we propose a symmetric optical flow method to address the well-known chicken-and-egg relation between optical flow and occlusions. In contrast to many state-of-the-art methods that consider occlusions as outliers, possibly filtered out during post-processing, we highlight the importance of joint occlusion reasoning in the optimization and show how to utilize occlusion as an important cue for estimating optical flow. The key feature of our model is to fully exploit the symmetry properties that characterize optical flow and occlusions in the two consecutive images. Specifically through utilizing forward-backward consistency and occlusion-disocclusion symmetry in the energy, our model jointly estimates optical flow in both forward and backward direction, as well as consistent occlusion maps in both views. We demonstrate significant performance benefits on standard benchmarks, especially from the occlusion-disocclusion symmetry. On the challenging KITTI dataset we report the most accurate two-frame results to date.