Learning Optical Flow Field via Neural Ordinary Differential Equation

TL;DR

This paper introduces a neural ODE-based method for optical flow estimation that dynamically adjusts refinement steps, achieving better accuracy with only one step compared to traditional fixed-step recurrent models.

Contribution

It presents a novel neural ODE approach for optical flow that models iterative refinement as a continuous process, improving flexibility and performance over fixed-step methods.

Findings

Achieves higher accuracy than baseline models.

Requires only a single refinement step.

Demonstrates flexibility of neural ODEs in optical flow tasks.

Abstract

Recent works on optical flow estimation use neural networks to predict the flow field that maps positions of one image to positions of the other. These networks consist of a feature extractor, a correlation volume, and finally several refinement steps. These refinement steps mimic the iterative refinements performed by classical optimization algorithms and are usually implemented by neural layers (e.g., GRU) which are recurrently executed for a fixed and pre-determined number of steps. However, relying on a fixed number of steps may result in suboptimal performance because it is not tailored to the input data. In this paper, we introduce a novel approach for predicting the derivative of the flow using a continuous model, namely neural ordinary differential equations (ODE). One key advantage of this approach is its capacity to model an equilibrium process, dynamically adjusting the number of compute steps based on the data at hand. By following a particular neural architecture, ODE solver, and associated hyperparameters, our proposed model can replicate the exact same updates as recurrent cells used in existing works, offering greater generality. Through extensive experimental analysis on optical flow benchmarks, we demonstrate that our approach achieves an impressive improvement over baseline and existing models, all while requiring only a single refinement step.