DCVNet: Dilated Cost Volume Networks for Fast Optical Flow

TL;DR

DCVNet introduces dilated cost volumes with a U-Net architecture for efficient, real-time optical flow estimation, matching accuracy of existing methods without sequential processing.

Contribution

It proposes a novel cost volume construction using dilation factors, enabling single-pass, real-time optical flow estimation without coarse-to-fine strategies.

Findings

Achieves 30 fps inference on a mid-end GPU.

Provides comparable accuracy to state-of-the-art methods.

Simplifies the optical flow pipeline with single-pass processing.

Abstract

The cost volume, capturing the similarity of possible correspondences across two input images, is a key ingredient in state-of-the-art optical flow approaches. When sampling correspondences to build the cost volume, a large neighborhood radius is required to deal with large displacements, introducing a significant computational burden. To address this, coarse-to-fine or recurrent processing of the cost volume is usually adopted, where correspondence sampling in a local neighborhood with a small radius suffices. In this paper, we propose an alternative by constructing cost volumes with different dilation factors to capture small and large displacements simultaneously. A U-Net with skip connections is employed to convert the dilated cost volumes into interpolation weights between all possible captured displacements to get the optical flow. Our proposed model DCVNet only needs to process the cost volume once in a simple feedforward manner and does not rely on the sequential processing strategy. DCVNet obtains comparable accuracy to existing approaches and achieves real-time inference (30 fps on a mid-end 1080ti GPU). The code and model weights are available at https://github.com/neu-vi/ezflow.