MultiDepth: Multi-Sample Priors for Refining Monocular Metric Depth Estimations in Indoor Scenes

TL;DR

MultiDepth introduces a lightweight refinement method for monocular depth estimation in indoor scenes, leveraging multi-sample inputs to significantly improve accuracy while reducing computational overhead.

Contribution

It presents a novel multi-sample refinement approach that enhances depth estimation accuracy without normal map prediction, lowering model size and complexity.

Findings

Achieves over 45% accuracy improvement on multiple datasets.

Reduces model size and computation compared to existing methods.

Effectively refines initial depth predictions with minimal overhead.

Abstract

Monocular metric depth estimation (MMDE) is a crucial task to solve for indoor scene reconstruction on edge devices. Despite this importance, existing models are sensitive to factors such as boundary frequency of objects in the scene and scene complexity, failing to fully capture many indoor scenes. In this work, we propose to close this gap through the task of monocular metric depth refinement (MMDR) by leveraging state-of-the-art MMDE models. MultiDepth proposes a solution by taking samples of the image along with the initial depth map prediction made by a pre-trained MMDE model. Compared to existing iterative depth refinement techniques, MultiDepth does not employ normal map prediction as part of its architecture, effectively lowering the model size and computation overhead while outputting impactful changes from refining iterations. MultiDepth implements a lightweight encoder-decoder architecture for the refinement network, processing multiple samples from the given image, including segmentation masking. We evaluate MultiDepth on four datasets and compare them to state-of-the-art methods to demonstrate its effective refinement with minimal overhead, displaying accuracy improvement upward of 45%.