Self-Distilled Depth Refinement with Noisy Poisson Fusion

TL;DR

This paper introduces a noise-robust depth refinement framework called SDDR that uses self-distillation and edge guidance to improve high-resolution depth maps efficiently and with better boundary accuracy across multiple benchmarks.

Contribution

The paper proposes a novel self-distilled depth refinement method modeling the problem as noisy Poisson fusion, enhancing robustness, efficiency, and generalizability over existing patch-based approaches.

Findings

Significant accuracy improvements on five benchmarks.

Enhanced edge quality and boundary sharpness.

Increased efficiency and robustness in depth refinement.

Abstract

Depth refinement aims to infer high-resolution depth with fine-grained edges and details, refining low-resolution results of depth estimation models. The prevailing methods adopt tile-based manners by merging numerous patches, which lacks efficiency and produces inconsistency. Besides, prior arts suffer from fuzzy depth boundaries and limited generalizability. Analyzing the fundamental reasons for these limitations, we model depth refinement as a noisy Poisson fusion problem with local inconsistency and edge deformation noises. We propose the Self-distilled Depth Refinement (SDDR) framework to enforce robustness against the noises, which mainly consists of depth edge representation and edge-based guidance. With noisy depth predictions as input, SDDR generates low-noise depth edge representations as pseudo-labels by coarse-to-fine self-distillation. Edge-based guidance with edge-guided gradient loss and edge-based fusion loss serves as the optimization objective equivalent to Poisson fusion. When depth maps are better refined, the labels also become more noise-free. Our model can acquire strong robustness to the noises, achieving significant improvements in accuracy, edge quality, efficiency, and generalizability on five different benchmarks. Moreover, directly training another model with edge labels produced by SDDR brings improvements, suggesting that our method could help with training robust refinement models in future works.