Neural Disparity Refinement for Arbitrary Resolution Stereo

TL;DR

This paper presents a neural disparity refinement architecture that can produce high-quality disparity maps at any resolution, suitable for mobile devices with mixed camera resolutions, and can improve traditional stereo methods without retraining.

Contribution

A novel continuous formulation neural network that refines disparity maps at arbitrary resolutions and enhances traditional stereo algorithms with zero-shot generalization.

Findings

Effective disparity refinement at any resolution.

Improves traditional stereo methods like SGM.

Achieves zero-shot generalization performance.

Abstract

We introduce a novel architecture for neural disparity refinement aimed at facilitating deployment of 3D computer vision on cheap and widespread consumer devices, such as mobile phones. Our approach relies on a continuous formulation that enables to estimate a refined disparity map at any arbitrary output resolution. Thereby, it can handle effectively the unbalanced camera setup typical of nowadays mobile phones, which feature both high and low resolution RGB sensors within the same device. Moreover, our neural network can process seamlessly the output of a variety of stereo methods and, by refining the disparity maps computed by a traditional matching algorithm like SGM, it can achieve unpaired zero-shot generalization performance compared to state-of-the-art end-to-end stereo models.