FALDOI: A new minimization strategy for large displacement variational optical flow

TL;DR

FALDOI introduces a robust large displacement optical flow method that effectively utilizes sparse matches to guide energy minimization, outperforming traditional coarse-to-fine approaches in accuracy and robustness.

Contribution

The paper presents a novel minimization strategy that leverages sparse matches for large displacement optical flow, improving robustness and accuracy over existing methods.

Findings

Outperforms coarse-to-fine methods on benchmarks

Robust to sparse matches, noise, and outliers

Effectively captures large object displacements

Abstract

We propose a large displacement optical flow method that introduces a new strategy to compute a good local minimum of any optical flow energy functional. The method requires a given set of discrete matches, which can be extremely sparse, and an energy functional which locally guides the interpolation from those matches. In particular, the matches are used to guide a structured coordinate-descent of the energy functional around these keypoints. It results in a two-step minimization method at the finest scale which is very robust to the inevitable outliers of the sparse matcher and able to capture large displacements of small objects. Its benefits over other variational methods that also rely on a set of sparse matches are its robustness against very few matches, high levels of noise and outliers. We validate our proposal using several optical flow variational models. The results consistently outperform the coarse-to-fine approaches and achieve good qualitative and quantitative performance on the standard optical flow benchmarks.