AccFlow: Backward Accumulation for Long-Range Optical Flow

TL;DR

This paper introduces AccFlow, a recurrent framework that improves long-range optical flow estimation by backward accumulation of local flows, incorporating novel modules to reduce errors and handle occlusions, validated on a new large-scale dataset.

Contribution

The paper proposes a novel backward accumulation framework with adaptive modules for improved long-range optical flow estimation, demonstrating superiority over forward accumulation.

Findings

Backward accumulation outperforms forward accumulation.

AccFlow effectively handles complex object deformation and occlusion.

The new CVO dataset enables robust training and evaluation.

Abstract

Recent deep learning-based optical flow estimators have exhibited impressive performance in generating local flows between consecutive frames. However, the estimation of long-range flows between distant frames, particularly under complex object deformation and large motion occlusion, remains a challenging task. One promising solution is to accumulate local flows explicitly or implicitly to obtain the desired long-range flow. Nevertheless, the accumulation errors and flow misalignment can hinder the effectiveness of this approach. This paper proposes a novel recurrent framework called AccFlow, which recursively backward accumulates local flows using a deformable module called as AccPlus. In addition, an adaptive blending module is designed along with AccPlus to alleviate the occlusion effect by backward accumulation and rectify the accumulation error. Notably, we demonstrate the superiority of backward accumulation over conventional forward accumulation, which to the best of our knowledge has not been explicitly established before. To train and evaluate the proposed AccFlow, we have constructed a large-scale high-quality dataset named CVO, which provides ground-truth optical flow labels between adjacent and distant frames. Extensive experiments validate the effectiveness of AccFlow in handling long-range optical flow estimation. Codes are available at https://github.com/mulns/AccFlow .