Cost Function Unrolling in Unsupervised Optical Flow

TL;DR

This paper introduces Cost Unrolling, a novel iterative scheme that creates a differentiable proxy for the Total Variation semi-norm, improving gradient accuracy and convergence in unsupervised optical flow training without extra computational cost.

Contribution

It proposes a new method called Cost Unrolling to better approximate the Total Variation semi-norm, enhancing training and prediction quality in unsupervised optical flow models.

Findings

Achieved up to 15.82% EPE reduction on occluded pixels.

Improved motion edge detection and sharper predictions.

Enhanced convergence without increasing computational complexity.

Abstract

Steepest descent algorithms, which are commonly used in deep learning, use the gradient as the descent direction, either as-is or after a direction shift using preconditioning. In many scenarios calculating the gradient is numerically hard due to complex or non-differentiable cost functions, specifically next to singular points. In this work we focus on the derivation of the Total Variation semi-norm commonly used in unsupervised cost functions. Specifically, we derive a differentiable proxy to the hard L1 smoothness constraint in a novel iterative scheme which we refer to as Cost Unrolling. Producing more accurate gradients during training, our method enables finer predictions of a given DNN model through improved convergence, without modifying its architecture or increasing computational complexity. We demonstrate our method in the unsupervised optical flow task. Replacing the L1 smoothness constraint with our unrolled cost during the training of a well known baseline, we report improved results on both MPI Sintel and KITTI 2015 unsupervised optical flow benchmarks. Particularly, we report EPE reduced by up to 15.82% on occluded pixels, where the smoothness constraint is dominant, enabling the detection of much sharper motion edges.