Adaptive Depth-converted-Scale Convolution for Self-supervised Monocular Depth Estimation

TL;DR

This paper introduces an adaptive convolution method that explicitly models object size changes due to depth variations to improve monocular depth estimation, achieving state-of-the-art results on KITTI.

Contribution

The paper proposes Depth-converted-Scale Convolution (DcSConv) and a fusion module, enhancing existing CNNs for better depth estimation by handling object size-depth relationships.

Findings

Achieves up to 11.6% SqRel reduction on KITTI benchmark.

DcSConv improves feature extraction by focusing on adaptive scale.

The method is compatible as a plug-and-play module with existing CNNs.

Abstract

Self-supervised monocular depth estimation (MDE) has received increasing interests in the last few years. The objects in the scene, including the object size and relationship among different objects, are the main clues to extract the scene structure. However, previous works lack the explicit handling of the changing sizes of the object due to the change of its depth. Especially in a monocular video, the size of the same object is continuously changed, resulting in size and depth ambiguity. To address this problem, we propose a Depth-converted-Scale Convolution (DcSConv) enhanced monocular depth estimation framework, by incorporating the prior relationship between the object depth and object scale to extract features from appropriate scales of the convolution receptive field. The proposed DcSConv focuses on the adaptive scale of the convolution filter instead of the local deformation of its shape. It establishes that the scale of the convolution filter matters no less (or even more in the evaluated task) than its local deformation. Moreover, a Depth-converted-Scale aware Fusion (DcS-F) is developed to adaptively fuse the DcSConv features and the conventional convolution features. Our DcSConv enhanced monocular depth estimation framework can be applied on top of existing CNN based methods as a plug-and-play module to enhance the conventional convolution block. Extensive experiments with different baselines have been conducted on the KITTI benchmark and our method achieves the best results with an improvement up to 11.6% in terms of SqRel reduction. Ablation study also validates the effectiveness of each proposed module.