All-day Depth Completion
Vadim Ezhov, Hyoungseob Park, Zhaoyang Zhang, Rishi Upadhyay, Howard, Zhang, Chethan Chinder Chandrappa, Achuta Kadambi, Yunhao Ba, Julie Dorsey,, Alex Wong

TL;DR
This paper introduces a multi-sensor depth completion method that combines synthetic data, sparse LiDAR points, and images to improve depth estimation across day and night conditions, especially in low-light regions.
Contribution
It presents SpaDe, a synthetic data-based approach for initial depth approximation, and an uncertainty-driven residual learning scheme for refinement, enhancing depth completion under varying illumination.
Findings
Achieved 11.65% average improvement over baselines in all-day scenarios.
Demonstrated 25% enhancement when augmenting existing methods with SpaDe.
Improved depth estimation accuracy in both daytime and nighttime scenes.
Abstract
We propose a method for depth estimation under different illumination conditions, i.e., day and night time. As photometry is uninformative in regions under low-illumination, we tackle the problem through a multi-sensor fusion approach, where we take as input an additional synchronized sparse point cloud (i.e., from a LiDAR) projected onto the image plane as a sparse depth map, along with a camera image. The crux of our method lies in the use of the abundantly available synthetic data to first approximate the 3D scene structure by learning a mapping from sparse to (coarse) dense depth maps along with their predictive uncertainty - we term this, SpaDe. In poorly illuminated regions where photometric intensities do not afford the inference of local shape, the coarse approximation of scene depth serves as a prior; the uncertainty map is then used with the image to guide refinement through…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Vision and Imaging · Robotics and Sensor-Based Localization · Human Pose and Action Recognition
MethodsSpatially-Adaptive Normalization
