Depth Completion using Plane-Residual Representation

TL;DR

This paper introduces a novel plane-residual representation for depth completion, improving accuracy and efficiency by combining depth plane classification with residual regression, reducing the regression burden and inference time.

Contribution

The paper proposes a new depth representation and network architecture that enhances depth completion accuracy and speed by decomposing depth prediction into classification and residual regression.

Findings

Achieved improved depth completion accuracy over previous methods.

Reduced inference time due to efficient plane-residual approach.

Demonstrated effectiveness on benchmark datasets.

Abstract

The basic framework of depth completion is to predict a pixel-wise dense depth map using very sparse input data. In this paper, we try to solve this problem in a more effective way, by reformulating the regression-based depth estimation problem into a combination of depth plane classification and residual regression. Our proposed approach is to initially densify sparse depth information by figuring out which plane a pixel should lie among a number of discretized depth planes, and then calculate the final depth value by predicting the distance from the specified plane. This will help the network to lessen the burden of directly regressing the absolute depth information from none, and to effectively obtain more accurate depth prediction result with less computation power and inference time. To do so, we firstly introduce a novel way of interpreting depth information with the closest depth plane label $p$ and a residual value $r$, as we call it, Plane-Residual (PR) representation. We also propose a depth completion network utilizing PR representation consisting of a shared encoder and two decoders, where one classifies the pixel's depth plane label, while the other one regresses the normalized distance from the classified depth plane. By interpreting depth information in PR representation and using our corresponding depth completion network, we were able to acquire improved depth completion performance with faster computation, compared to previous approaches.