Optical Flow Estimation via Motion Feature Recovery

TL;DR

This paper introduces a novel iterative Motion Feature Recovery method to address the Vanishing Cost Volume Problem in optical flow estimation, significantly improving performance on challenging scenarios with occlusion and large displacements.

Contribution

The paper proposes a new MFR approach that models motion consistency across multiple frames to recover lost motion features, enhancing optical flow accuracy.

Findings

Achieves state-of-the-art results on Sintel and KITTI datasets.

Ranks second on Sintel public leaderboard.

Effectively recovers invalid motion features for better flow estimation.

Abstract

Optical flow estimation with occlusion or large displacement is a problematic challenge due to the lost of corresponding pixels between consecutive frames. In this paper, we discover that the lost information is related to a large quantity of motion features (more than 40%) computed from the popular discriminative cost-volume feature would completely vanish due to invalid sampling, leading to the low efficiency of optical flow learning. We call this phenomenon the Vanishing Cost Volume Problem. Inspired by the fact that local motion tends to be highly consistent within a short temporal window, we propose a novel iterative Motion Feature Recovery (MFR) method to address the vanishing cost volume via modeling motion consistency across multiple frames. In each MFR iteration, invalid entries from original motion features are first determined based on the current flow. Then, an efficient network is designed to adaptively learn the motion correlation to recover invalid features for lost-information restoration. The final optical flow is then decoded from the recovered motion features. Experimental results on Sintel and KITTI show that our method achieves state-of-the-art performances. In fact, MFR currently ranks second on Sintel public website.