A Fusion Approach for Multi-Frame Optical Flow Estimation

TL;DR

This paper introduces a fusion-based multi-frame optical flow estimation method that leverages longer-term temporal cues, achieving state-of-the-art results on MPI Sintel and KITTI 2015 benchmarks.

Contribution

It proposes a simple fusion approach that combines optical flow estimates from multiple frames, enhancing accuracy over traditional two-frame methods.

Findings

Ranks first on MPI Sintel benchmark

Ranks first on KITTI 2015 benchmark

Models are publicly available on GitHub

Abstract

To date, top-performing optical flow estimation methods only take pairs of consecutive frames into account. While elegant and appealing, the idea of using more than two frames has not yet produced state-of-the-art results. We present a simple, yet effective fusion approach for multi-frame optical flow that benefits from longer-term temporal cues. Our method first warps the optical flow from previous frames to the current, thereby yielding multiple plausible estimates. It then fuses the complementary information carried by these estimates into a new optical flow field. At the time of writing, our method ranks first among published results in the MPI Sintel and KITTI 2015 benchmarks. Our models will be available on https://github.com/NVlabs/PWC-Net.