Geometry-Informed Distance Candidate Selection for Adaptive Lightweight Omnidirectional Stereo Vision with Fisheye Images

TL;DR

This paper introduces a geometry-informed method for selecting distance candidates in omnidirectional stereo vision, significantly reducing computational costs and improving accuracy without retraining, suitable for resource-limited mobile robots.

Contribution

A novel geometry-informed distance candidate selection approach that adapts during deployment, enhancing accuracy and efficiency in omnidirectional stereo vision systems.

Findings

Reduces computational cost by using fewer candidates.

Improves accuracy without retraining or fine-tuning.

Outperforms evenly distributed candidate models.

Abstract

Multi-view stereo omnidirectional distance estimation usually needs to build a cost volume with many hypothetical distance candidates. The cost volume building process is often computationally heavy considering the limited resources a mobile robot has. We propose a new geometry-informed way of distance candidates selection method which enables the use of a very small number of candidates and reduces the computational cost. We demonstrate the use of the geometry-informed candidates in a set of model variants. We find that by adjusting the candidates during robot deployment, our geometry-informed distance candidates also improve a pre-trained model's accuracy if the extrinsics or the number of cameras changes. Without any re-training or fine-tuning, our models outperform models trained with evenly distributed distance candidates. Models are also released as hardware-accelerated versions with a new dedicated large-scale dataset. The project page, code, and dataset can be found at https://theairlab.org/gicandidates/ .