HMAFlow: Learning More Accurate Optical Flow via Hierarchical Motion Field Alignment

TL;DR

HMAFlow introduces a hierarchical motion field alignment and correlation self-attention to enhance optical flow accuracy, especially for small objects, achieving state-of-the-art results on Sintel and KITTI benchmarks.

Contribution

The paper proposes HMAFlow, a novel optical flow model with hierarchical alignment and multi-scale correlation search, improving accuracy over existing methods.

Findings

Achieves 14.2% and 3.4% error reduction on Sintel benchmarks.

Surpasses RAFT and GMA on KITTI Fl-all metric.

Demonstrates superior generalization performance.

Abstract

Optical flow estimation is a fundamental and long-standing visual task. In this work, we present a novel method, dubbed HMAFlow, to improve optical flow estimation in challenging scenes, particularly those involving small objects. The proposed model mainly consists of two core components: a Hierarchical Motion Field Alignment (HMA) module and a Correlation Self-Attention (CSA) module. In addition, we rebuild 4D cost volumes by employing a Multi-Scale Correlation Search (MCS) layer and replacing average pooling in common cost volumes with a search strategy utilizing multiple search ranges. Experimental results demonstrate that our model achieves the best generalization performance compared to other state-of-the-art methods. Specifically, compared with RAFT, our method achieves relative error reductions of 14.2% and 3.4% on the clean pass and final pass of the Sintel online benchmark, respectively. On the KITTI test benchmark, HMAFlow surpasses RAFT and GMA in the Fl-all metric by relative margins of 6.8% and 7.7%, respectively. To facilitate future research, our code will be made available at https://github.com/BooTurbo/HMAFlow.