Plane2Depth: Hierarchical Adaptive Plane Guidance for Monocular Depth Estimation

TL;DR

Plane2Depth introduces a hierarchical framework that adaptively incorporates plane information to significantly enhance monocular depth estimation, especially in low-texture indoor scenes, outperforming existing methods.

Contribution

The paper presents a novel plane-guided depth generator and adaptive feature aggregation modules that effectively utilize scene planes for improved depth prediction.

Findings

Outperforms state-of-the-art on NYU-Depth-v2

Achieves competitive results on KITTI

Generalizes well to unseen scenes

Abstract

Monocular depth estimation aims to infer a dense depth map from a single image, which is a fundamental and prevalent task in computer vision. Many previous works have shown impressive depth estimation results through carefully designed network structures, but they usually ignore the planar information and therefore perform poorly in low-texture areas of indoor scenes. In this paper, we propose Plane2Depth, which adaptively utilizes plane information to improve depth prediction within a hierarchical framework. Specifically, in the proposed plane guided depth generator (PGDG), we design a set of plane queries as prototypes to softly model planes in the scene and predict per-pixel plane coefficients. Then the predicted plane coefficients can be converted into metric depth values with the pinhole camera model. In the proposed adaptive plane query aggregation (APGA) module, we introduce a novel feature interaction approach to improve the aggregation of multi-scale plane features in a top-down manner. Extensive experiments show that our method can achieve outstanding performance, especially in low-texture or repetitive areas. Furthermore, under the same backbone network, our method outperforms the state-of-the-art methods on the NYU-Depth-v2 dataset, achieves competitive results with state-of-the-art methods KITTI dataset and can be generalized to unseen scenes effectively.