Multi-Hypothesis Visual-Inertial Flow

TL;DR

This paper introduces VIFlow, an unsupervised deep neural network that estimates pixel correspondences by generating multiple hypotheses using visual and inertial data, improving efficiency and enabling anomaly detection.

Contribution

The paper presents VIFlow, a novel multi-hypothesis unsupervised neural network architecture that combines visual and inertial data for dense correspondence estimation.

Findings

VIFlow achieves comparable accuracy to state-of-the-art methods.

It significantly reduces runtime and increases efficiency.

VIFlow can detect anomalous independent motion.

Abstract

Estimating the correspondences between pixels in sequences of images is a critical first step for a myriad of tasks including vision-aided navigation (e.g., visual odometry (VO), visual-inertial odometry (VIO), and visual simultaneous localization and mapping (VSLAM)) and anomaly detection. We introduce a new unsupervised deep neural network architecture called the Visual Inertial Flow (VIFlow) network and demonstrate image correspondence and optical flow estimation by an unsupervised multi-hypothesis deep neural network receiving grayscale imagery and extra-visual inertial measurements. VIFlow learns to combine heterogeneous sensor streams and sample from an unknown, un-parametrized noise distribution to generate several (4 or 8 in this work) probable hypotheses on the pixel-level correspondence mappings between a source image and a target image . We quantitatively benchmark VIFlow against several leading vision-only dense correspondence and flow methods and show a substantial decrease in runtime and increase in efficiency compared to all methods with similar performance to state-of-the-art (SOA) dense correspondence matching approaches. We also present qualitative results showing how VIFlow can be used for detecting anomalous independent motion.