A benchmark with decomposed distribution shifts for 360 monocular depth estimation

TL;DR

This paper introduces a benchmark for monocular depth estimation that decomposes distribution shifts into covariate, prior, and concept shifts, highlighting their unique challenges and the compounded difficulty when combined.

Contribution

It presents a novel benchmark that systematically analyzes different types of distribution shifts in monocular depth estimation, emphasizing their distinct and combined effects.

Findings

Each distribution shift type presents unique challenges.

Combining shifts leads to greater performance drops.

Standard approaches struggle with combined shifts.

Abstract

In this work we contribute a distribution shift benchmark for a computer vision task; monocular depth estimation. Our differentiation is the decomposition of the wider distribution shift of uncontrolled testing on in-the-wild data, to three distinct distribution shifts. Specifically, we generate data via synthesis and analyze them to produce covariate (color input), prior (depth output) and concept (their relationship) distribution shifts. We also synthesize combinations and show how each one is indeed a different challenge to address, as stacking them produces increased performance drops and cannot be addressed horizontally using standard approaches.