Semantically Guided Depth Upsampling

TL;DR

This paper introduces a new depth upsampling method that combines semantic scene labeling and boundary cues within a geodesic framework to produce accurate, boundary-preserving dense depth maps from sparse data.

Contribution

It presents a novel approach that integrates semantic and boundary information for depth upsampling, improving accuracy and detail preservation over existing methods.

Findings

Outperforms state-of-the-art methods on multiple datasets.

Effectively preserves object boundaries and fine details.

Handles very sparse depth measurements robustly.

Abstract

We present a novel method for accurate and efficient up- sampling of sparse depth data, guided by high-resolution imagery. Our approach goes beyond the use of intensity cues only and additionally exploits object boundary cues through structured edge detection and semantic scene labeling for guidance. Both cues are combined within a geodesic distance measure that allows for boundary-preserving depth in- terpolation while utilizing local context. We model the observed scene structure by locally planar elements and formulate the upsampling task as a global energy minimization problem. Our method determines glob- ally consistent solutions and preserves fine details and sharp depth bound- aries. In our experiments on several public datasets at different levels of application, we demonstrate superior performance of our approach over the state-of-the-art, even for very sparse measurements.