Joint Depth and Normal Estimation from Real-world Time-of-flight Raw Data

TL;DR

This paper introduces a new method for jointly estimating depth and surface normals from raw ToF sensor data, supported by a large-scale dataset and a robust learning framework, leading to superior reconstruction quality.

Contribution

The paper presents the first large-scale ToF dataset with ground-truth depth, and a novel joint estimation framework that outperforms existing methods.

Findings

Significantly improves depth and normal map quality.

Outperforms state-of-the-art approaches.

Provides a large-scale dataset for ToF data analysis.

Abstract

We present a novel approach to joint depth and normal estimation for time-of-flight (ToF) sensors. Our model learns to predict the high-quality depth and normal maps jointly from ToF raw sensor data. To achieve this, we meticulously constructed the first large-scale dataset (named ToF-100) with paired raw ToF data and ground-truth high-resolution depth maps provided by an industrial depth camera. In addition, we also design a simple but effective framework for joint depth and normal estimation, applying a robust Chamfer loss via jittering to improve the performance of our model. Our experiments demonstrate that our proposed method can efficiently reconstruct high-resolution depth and normal maps and significantly outperforms state-of-the-art approaches. Our code and data will be available at \url{https://github.com/hkustVisionRr/JointlyDepthNormalEstimation}