Flow Fields: Dense Correspondence Fields for Highly Accurate Large Displacement Optical Flow Estimation

TL;DR

This paper introduces a novel dense correspondence field method for large displacement optical flow that reduces outliers without explicit regularization, significantly improving accuracy over existing descriptor matching techniques.

Contribution

The authors propose a new data-based search strategy for dense correspondence fields that is less prone to outliers and enhances large displacement optical flow estimation.

Findings

Outperforms state-of-the-art descriptor matching techniques in optical flow accuracy.

Significantly improves EpicFlow results on MPI-Sintel, KITTI, and Middlebury datasets.

Does not require explicit regularization or smoothing, simplifying the process.

Abstract

Modern large displacement optical flow algorithms usually use an initialization by either sparse descriptor matching techniques or dense approximate nearest neighbor fields. While the latter have the advantage of being dense, they have the major disadvantage of being very outlier prone as they are not designed to find the optical flow, but the visually most similar correspondence. In this paper we present a dense correspondence field approach that is much less outlier prone and thus much better suited for optical flow estimation than approximate nearest neighbor fields. Our approach is conceptually novel as it does not require explicit regularization, smoothing (like median filtering) or a new data term, but solely our novel purely data based search strategy that finds most inliers (even for small objects), while it effectively avoids finding outliers. Moreover, we present novel enhancements for outlier filtering. We show that our approach is better suited for large displacement optical flow estimation than state-of-the-art descriptor matching techniques. We do so by initializing EpicFlow (so far the best method on MPI-Sintel) with our Flow Fields instead of their originally used state-of-the-art descriptor matching technique. We significantly outperform the original EpicFlow on MPI-Sintel, KITTI and Middlebury.