Joint Unsupervised Learning of Optical Flow and Egomotion with Bi-Level Optimization

TL;DR

This paper introduces a novel unsupervised deep learning framework that jointly estimates optical flow and camera motion in rigid scenes by embedding geometric constraints via bi-level optimization, leading to improved accuracy.

Contribution

It formulates joint optical flow and egomotion estimation as a bi-level optimization problem with implicit differentiation, enabling end-to-end training with geometric constraints.

Findings

Improved optical flow estimation in challenging scenarios.

More accurate camera motion estimates than previous unsupervised methods.

Effective integration of epipolar geometry into deep learning framework.

Abstract

We address the problem of joint optical flow and camera motion estimation in rigid scenes by incorporating geometric constraints into an unsupervised deep learning framework. Unlike existing approaches which rely on brightness constancy and local smoothness for optical flow estimation, we exploit the global relationship between optical flow and camera motion using epipolar geometry. In particular, we formulate the prediction of optical flow and camera motion as a bi-level optimization problem, consisting of an upper-level problem to estimate the flow that conforms to the predicted camera motion, and a lower-level problem to estimate the camera motion given the predicted optical flow. We use implicit differentiation to enable back-propagation through the lower-level geometric optimization layer independent of its implementation, allowing end-to-end training of the network. With globally-enforced geometric constraints, we are able to improve the quality of the estimated optical flow in challenging scenarios and obtain better camera motion estimates compared to other unsupervised learning methods.