Context-Aware Iteration Policy Network for Efficient Optical Flow Estimation

TL;DR

This paper introduces a context-aware iteration policy network that dynamically determines the optimal number of iterations for optical flow estimation, significantly reducing computational costs while maintaining accuracy.

Contribution

It proposes a novel policy network that learns to skip unnecessary iterations based on contextual information, improving efficiency of optical flow networks.

Findings

Reduces FLOPs by about 40% on Sintel dataset

Reduces FLOPs by about 20% on KITTI dataset

Maintains comparable optical flow estimation performance

Abstract

Existing recurrent optical flow estimation networks are computationally expensive since they use a fixed large number of iterations to update the flow field for each sample. An efficient network should skip iterations when the flow improvement is limited. In this paper, we develop a Context-Aware Iteration Policy Network for efficient optical flow estimation, which determines the optimal number of iterations per sample. The policy network achieves this by learning contextual information to realize whether flow improvement is bottlenecked or minimal. On the one hand, we use iteration embedding and historical hidden cell, which include previous iterations information, to convey how flow has changed from previous iterations. On the other hand, we use the incremental loss to make the policy network implicitly perceive the magnitude of optical flow improvement in the subsequent iteration. Furthermore, the computational complexity in our dynamic network is controllable, allowing us to satisfy various resource preferences with a single trained model. Our policy network can be easily integrated into state-of-the-art optical flow networks. Extensive experiments show that our method maintains performance while reducing FLOPs by about 40%/20% for the Sintel/KITTI datasets.