LIFE: Lighting Invariant Flow Estimation

TL;DR

LIFE introduces a weakly supervised neural network framework that estimates lighting-invariant optical flow between image pairs with large lighting variations and viewpoint changes, improving robustness in challenging scenarios.

Contribution

The paper presents a novel weakly supervised approach guiding flow estimation with structure-from-motion, enhancing performance under lighting and viewpoint variations.

Findings

LIFE outperforms previous flow estimation methods in challenging scenarios.

Guided feature matching improves correspondence accuracy.

Enhanced flow estimation benefits downstream tasks.

Abstract

We tackle the problem of estimating flow between two images with large lighting variations. Recent learning-based flow estimation frameworks have shown remarkable performance on image pairs with small displacement and constant illuminations, but cannot work well on cases with large viewpoint change and lighting variations because of the lack of pixel-wise flow annotations for such cases. We observe that via the Structure-from-Motion (SfM) techniques, one can easily estimate relative camera poses between image pairs with large viewpoint change and lighting variations. We propose a novel weakly supervised framework LIFE to train a neural network for estimating accurate lighting-invariant flows between image pairs. Sparse correspondences are conventionally established via feature matching with descriptors encoding local image contents. However, local image contents are inevitably ambiguous and error-prone during the cross-image feature matching process, which hinders downstream tasks. We propose to guide feature matching with the flows predicted by LIFE, which addresses the ambiguous matching by utilizing abundant context information in the image pairs. We show that LIFE outperforms previous flow learning frameworks by large margins in challenging scenarios, consistently improves feature matching, and benefits downstream tasks.