ProbFlow: Joint Optical Flow and Uncertainty Estimation

TL;DR

ProbFlow introduces a unified probabilistic framework that jointly estimates optical flow and its uncertainty, providing more reliable and accurate flow predictions along with a meaningful confidence measure.

Contribution

It presents a novel variational inference method that integrates flow estimation and uncertainty prediction, outperforming existing post-hoc confidence measures.

Findings

Achieves competitive optical flow results on benchmarks.

Provides a more reliable uncertainty measure than previous methods.

Demonstrates flexibility across different energy formulations.

Abstract

Optical flow estimation remains challenging due to untextured areas, motion boundaries, occlusions, and more. Thus, the estimated flow is not equally reliable across the image. To that end, post-hoc confidence measures have been introduced to assess the per-pixel reliability of the flow. We overcome the artificial separation of optical flow and confidence estimation by introducing a method that jointly predicts optical flow and its underlying uncertainty. Starting from common energy-based formulations, we rely on the corresponding posterior distribution of the flow given the images. We derive a variational inference scheme based on mean field, which incorporates best practices from energy minimization. An uncertainty measure is obtained along the flow at every pixel as the (marginal) entropy of the variational distribution. We demonstrate the flexibility of our probabilistic approach by applying it to two different energies and on two benchmarks. We not only obtain flow results that are competitive with the underlying energy minimization approach, but also a reliable uncertainty measure that significantly outperforms existing post-hoc approaches.