Unsupervised convolutional neural networks for motion estimation

TL;DR

This paper introduces an unsupervised CNN approach for dense motion estimation between image pairs, leveraging a differentiable optical flow constraint for training without ground truth data, achieving competitive results.

Contribution

It presents a novel unsupervised training method for CNN-based motion estimation using a differentiable optical flow constraint, eliminating the need for large labeled datasets.

Findings

Performs comparably to state-of-the-art methods on synthetic and real data.

Uses a differentiable cost function based on optical flow constraints for training.

Enables training of CNNs for motion estimation without ground truth labels.

Abstract

Traditional methods for motion estimation estimate the motion field F between a pair of images as the one that minimizes a predesigned cost function. In this paper, we propose a direct method and train a Convolutional Neural Network (CNN) that when, at test time, is given a pair of images as input it produces a dense motion field F at its output layer. In the absence of large datasets with ground truth motion that would allow classical supervised training, we propose to train the network in an unsupervised manner. The proposed cost function that is optimized during training, is based on the classical optical flow constraint. The latter is differentiable with respect to the motion field and, therefore, allows backpropagation of the error to previous layers of the network. Our method is tested on both synthetic and real image sequences and performs similarly to the state-of-the-art methods.