Deep Optical Flow Estimation Via Multi-Scale Correspondence Structure Learning

TL;DR

This paper introduces an end-to-end deep learning method called MSCSL for optical flow estimation, which effectively models multi-scale correspondence structures and their dependencies using a novel Conv-GRU network, achieving strong results on benchmarks.

Contribution

The paper proposes a unified multi-scale correspondence structure learning framework with a spatial Conv-GRU for adaptive dependency modeling in optical flow estimation.

Findings

Effective multi-scale correspondence structure modeling

Adaptive dependency modeling with Conv-GRU

Superior performance on benchmark datasets

Abstract

As an important and challenging problem in computer vision, learning based optical flow estimation aims to discover the intrinsic correspondence structure between two adjacent video frames through statistical learning. Therefore, a key issue to solve in this area is how to effectively model the multi-scale correspondence structure properties in an adaptive end-to-end learning fashion. Motivated by this observation, we propose an end-to-end multi-scale correspondence structure learning (MSCSL) approach for optical flow estimation. In principle, the proposed MSCSL approach is capable of effectively capturing the multi-scale inter-image-correlation correspondence structures within a multi-level feature space from deep learning. Moreover, the proposed MSCSL approach builds a spatial Conv-GRU neural network model to adaptively model the intrinsic dependency relationships among these multi-scale correspondence structures. Finally, the above procedures for correspondence structure learning and multi-scale dependency modeling are implemented in a unified end-to-end deep learning framework. Experimental results on several benchmark datasets demonstrate the effectiveness of the proposed approach.