FastFlowNet: A Lightweight Network for Fast Optical Flow Estimation

TL;DR

FastFlowNet is a lightweight, fast optical flow estimation network that achieves high accuracy with significantly reduced computational cost, suitable for low-power devices like mobile phones.

Contribution

The paper introduces novel modules such as HEPP, CDDC, and SBD to create a compact, efficient optical flow network with minimal accuracy loss.

Findings

Requires only 1/10 of the computation of comparable networks

Contains 1.37 million parameters

Achieves 90 FPS on GTX 1080Ti and 5.7 FPS on Jetson TX2

Abstract

Dense optical flow estimation plays a key role in many robotic vision tasks. In the past few years, with the advent of deep learning, we have witnessed great progress in optical flow estimation. However, current networks often consist of a large number of parameters and require heavy computation costs, largely hindering its application on low power-consumption devices such as mobile phones. In this paper, we tackle this challenge and design a lightweight model for fast and accurate optical flow prediction. Our proposed FastFlowNet follows the widely-used coarse-to-fine paradigm with following innovations. First, a new head enhanced pooling pyramid (HEPP) feature extractor is employed to intensify high-resolution pyramid features while reducing parameters. Second, we introduce a new center dense dilated correlation (CDDC) layer for constructing compact cost volume that can keep large search radius with reduced computation burden. Third, an efficient shuffle block decoder (SBD) is implanted into each pyramid level to accelerate flow estimation with marginal drops in accuracy. Experiments on both synthetic Sintel data and real-world KITTI datasets demonstrate the effectiveness of the proposed approach, which needs only 1/10 computation of comparable networks to achieve on par accuracy. In particular, FastFlowNet only contains 1.37M parameters; and can execute at 90 FPS (with a single GTX 1080Ti) or 5.7 FPS (embedded Jetson TX2 GPU) on a pair of Sintel images of resolution 1024x436.