Hybrid Cost Volume for Memory-Efficient Optical Flow

TL;DR

This paper introduces HCV, a memory-efficient hybrid cost volume for optical flow estimation that reduces memory usage significantly while maintaining high accuracy, especially for high-resolution images.

Contribution

The paper proposes a novel hybrid cost volume construction using a Top-k strategy and local search, enabling memory-efficient optical flow estimation with high accuracy.

Findings

Reduces memory consumption compared to all-pairs cost volume methods.

Achieves high accuracy on Sintel and KITTI datasets.

Effectively handles 4K resolution images.

Abstract

Current state-of-the-art flow methods are mostly based on dense all-pairs cost volumes. However, as image resolution increases, the computational and spatial complexity of constructing these cost volumes grows at a quartic rate, making these methods impractical for high-resolution images. In this paper, we propose a novel Hybrid Cost Volume for memory-efficient optical flow, named HCV. To construct HCV, we first propose a Top-k strategy to separate the 4D cost volume into two global 3D cost volumes. These volumes significantly reduce memory usage while retaining a substantial amount of matching information. We further introduce a local 4D cost volume with a local search space to supplement the local information for HCV. Based on HCV, we design a memory-efficient optical flow network, named HCVFlow. Compared to the recurrent flow methods based the all-pairs cost volumes, our HCVFlow significantly reduces memory consumption while ensuring high accuracy. We validate the effectiveness and efficiency of our method on the Sintel and KITTI datasets and real-world 4K (2160*3840) resolution images. Extensive experiments show that our HCVFlow has very low memory usage and outperforms other memory-efficient methods in terms of accuracy. The code is publicly available at https://github.com/gangweiX/HCVFlow.