FlowNAS: Neural Architecture Search for Optical Flow Estimation

TL;DR

FlowNAS introduces a neural architecture search method to optimize encoder architectures specifically for optical flow estimation, outperforming handcrafted models in accuracy and efficiency.

Contribution

The paper presents a novel neural architecture search framework tailored for optical flow estimation, including a new search space, Feature Alignment Distillation, and an evolutionary algorithm.

Findings

Achieves 4.67% F1-all error on KITTI, surpassing state-of-the-art models.

Reduces model complexity and latency compared to handcrafted architectures.

Demonstrates the effectiveness of automated architecture search for flow estimation.

Abstract

Existing optical flow estimators usually employ the network architectures typically designed for image classification as the encoder to extract per-pixel features. However, due to the natural difference between the tasks, the architectures designed for image classification may be sub-optimal for flow estimation. To address this issue, we propose a neural architecture search method named FlowNAS to automatically find the better encoder architecture for flow estimation task. We first design a suitable search space including various convolutional operators and construct a weight-sharing super-network for efficiently evaluating the candidate architectures. Then, for better training the super-network, we propose Feature Alignment Distillation, which utilizes a well-trained flow estimator to guide the training of super-network. Finally, a resource-constrained evolutionary algorithm is exploited to find an optimal architecture (i.e., sub-network). Experimental results show that the discovered architecture with the weights inherited from the super-network achieves 4.67\% F1-all error on KITTI, an 8.4\% reduction of RAFT baseline, surpassing state-of-the-art handcrafted models GMA and AGFlow, while reducing the model complexity and latency. The source code and trained models will be released in https://github.com/VDIGPKU/FlowNAS.