Expansion of Visual Hints for Improved Generalization in Stereo Matching

TL;DR

This paper proposes a novel method that expands sparse 3D visual hints using a 3D geometric graph to improve stereo matching generalization, demonstrating effectiveness on benchmarks and embedded hardware.

Contribution

It introduces a 3D hints expansion technique for stereo matching, inspired by Visual Inertial Odometry, enhancing robustness without extra sensors.

Findings

Improved stereo matching performance on multiple benchmarks.

Effective hints expansion without additional sensors.

Method runs efficiently on embedded hardware.

Abstract

We introduce visual hints expansion for guiding stereo matching to improve generalization. Our work is motivated by the robustness of Visual Inertial Odometry (VIO) in computer vision and robotics, where a sparse and unevenly distributed set of feature points characterizes a scene. To improve stereo matching, we propose to elevate 2D hints to 3D points. These sparse and unevenly distributed 3D visual hints are expanded using a 3D random geometric graph, which enhances the learning and inference process. We evaluate our proposal on multiple widely adopted benchmarks and show improved performance without access to additional sensors other than the image sequence. To highlight practical applicability and symbiosis with visual odometry, we demonstrate how our methods run on embedded hardware.