BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimation

TL;DR

BinsFormer introduces a novel Transformer-based framework for monocular depth estimation that adaptively generates bins and enhances spatial understanding, achieving state-of-the-art results on multiple datasets.

Contribution

The paper proposes a new method using Transformer decoders for adaptive bin generation and multi-scale decoding for improved depth estimation accuracy.

Findings

Outperforms existing methods on KITTI, NYU, and SUN RGB-D datasets.

Uses a novel set-to-set prediction approach for bin generation.

Incorporates scene understanding queries to enhance depth accuracy.

Abstract

Monocular depth estimation is a fundamental task in computer vision and has drawn increasing attention. Recently, some methods reformulate it as a classification-regression task to boost the model performance, where continuous depth is estimated via a linear combination of predicted probability distributions and discrete bins. In this paper, we present a novel framework called BinsFormer, tailored for the classification-regression-based depth estimation. It mainly focuses on two crucial components in the specific task: 1) proper generation of adaptive bins and 2) sufficient interaction between probability distribution and bins predictions. To specify, we employ the Transformer decoder to generate bins, novelly viewing it as a direct set-to-set prediction problem. We further integrate a multi-scale decoder structure to achieve a comprehensive understanding of spatial geometry information and estimate depth maps in a coarse-to-fine manner. Moreover, an extra scene understanding query is proposed to improve the estimation accuracy, which turns out that models can implicitly learn useful information from an auxiliary environment classification task. Extensive experiments on the KITTI, NYU, and SUN RGB-D datasets demonstrate that BinsFormer surpasses state-of-the-art monocular depth estimation methods with prominent margins. Code and pretrained models will be made publicly available at \url{https://github.com/zhyever/Monocular-Depth-Estimation-Toolbox}.