Optical Flow Training under Limited Label Budget via Active Learning

TL;DR

This paper introduces an active learning approach for optical flow training that significantly reduces annotation costs while maintaining high accuracy, demonstrating effectiveness on synthetic and real datasets.

Contribution

It proposes a semi-supervised training method combined with simple heuristic-based active learning to minimize labeling effort in optical flow prediction.

Findings

Semi-supervised training with 50% labels approaches full-label accuracy.

Active learning reduces label requirements to about 20% for similar accuracy.

Insights into factors affecting active learning performance are provided.

Abstract

Supervised training of optical flow predictors generally yields better accuracy than unsupervised training. However, the improved performance comes at an often high annotation cost. Semi-supervised training trades off accuracy against annotation cost. We use a simple yet effective semi-supervised training method to show that even a small fraction of labels can improve flow accuracy by a significant margin over unsupervised training. In addition, we propose active learning methods based on simple heuristics to further reduce the number of labels required to achieve the same target accuracy. Our experiments on both synthetic and real optical flow datasets show that our semi-supervised networks generally need around 50% of the labels to achieve close to full-label accuracy, and only around 20% with active learning on Sintel. We also analyze and show insights on the factors that may influence active learning performance. Code is available at https://github.com/duke-vision/optical-flow-active-learning-release.