MEDeA: Multi-view Efficient Depth Adjustment

TL;DR

MEDeA is a fast, multi-view depth adjustment method that improves depth consistency across frames using only RGB data, outperforming existing approaches on multiple benchmarks.

Contribution

It introduces a novel, efficient multi-view test-time depth adjustment technique that does not require additional data like normals or optical flow.

Findings

Sets new state-of-the-art on TUM RGB-D, 7Scenes, and ScanNet.

Achieves an order of magnitude faster processing than existing methods.

Successfully handles smartphone-captured data from ARKitScenes.

Abstract

The majority of modern single-view depth estimation methods predict relative depth and thus cannot be directly applied in many real-world scenarios, despite impressive performance in the benchmarks. Moreover, single-view approaches cannot guarantee consistency across a sequence of frames. Consistency is typically addressed with test-time optimization of discrepancy across views; however, it takes hours to process a single scene. In this paper, we present MEDeA, an efficient multi-view test-time depth adjustment method, that is an order of magnitude faster than existing test-time approaches. Given RGB frames with camera parameters, MEDeA predicts initial depth maps, adjusts them by optimizing local scaling coefficients, and outputs temporally-consistent depth maps. Contrary to test-time methods requiring normals, optical flow, or semantics estimation, MEDeA produces high-quality predictions with a depth estimation network solely. Our method sets a new state-of-the-art on TUM RGB-D, 7Scenes, and ScanNet benchmarks and successfully handles smartphone-captured data from ARKitScenes dataset.