PatchBatch: a Batch Augmented Loss for Optical Flow

TL;DR

This paper introduces PatchBatch, a novel deep learning pipeline for optical flow that uses a Siamese CNN for descriptor extraction, a unique loss function, and achieves state-of-the-art results on benchmarks.

Contribution

It presents a new loss function based on higher moments of loss distributions, improving optical flow accuracy with a Siamese CNN and efficient patch matching.

Findings

Achieves state-of-the-art performance on optical flow benchmarks.

Introduces a novel loss function based on higher moments.

Employs an efficient patch matching technique.

Abstract

We propose a new pipeline for optical flow computation, based on Deep Learning techniques. We suggest using a Siamese CNN to independently, and in parallel, compute the descriptors of both images. The learned descriptors are then compared efficiently using the L2 norm and do not require network processing of patch pairs. The success of the method is based on an innovative loss function that computes higher moments of the loss distributions for each training batch. Combined with an Approximate Nearest Neighbor patch matching method and a flow interpolation technique, state of the art performance is obtained on the most challenging and competitive optical flow benchmarks.