Semi-Supervised Learning of Optical Flow by Flow Supervisor

TL;DR

This paper introduces a novel semi-supervised fine-tuning approach for optical flow CNNs using a flow supervisor for self-supervision, enabling adaptation without ground truth flows and improving performance on benchmarks.

Contribution

It presents a new flow supervisor design for stable self-supervision, facilitating effective fine-tuning without ground truth flows, and achieves state-of-the-art results.

Findings

Effective semi-supervised fine-tuning on target datasets

Improved accuracy over existing self-supervision methods

State-of-the-art results on Sintel and KITTI benchmarks

Abstract

A training pipeline for optical flow CNNs consists of a pretraining stage on a synthetic dataset followed by a fine tuning stage on a target dataset. However, obtaining ground truth flows from a target video requires a tremendous effort. This paper proposes a practical fine tuning method to adapt a pretrained model to a target dataset without ground truth flows, which has not been explored extensively. Specifically, we propose a flow supervisor for self-supervision, which consists of parameter separation and a student output connection. This design is aimed at stable convergence and better accuracy over conventional self-supervision methods which are unstable on the fine tuning task. Experimental results show the effectiveness of our method compared to different self-supervision methods for semi-supervised learning. In addition, we achieve meaningful improvements over state-of-the-art optical flow models on Sintel and KITTI benchmarks by exploiting additional unlabeled datasets. Code is available at https://github.com/iwbn/flow-supervisor.