Depth Completion in Unseen Field Robotics Environments Using Extremely Sparse Depth Measurements

TL;DR

This paper introduces a depth completion model trained on synthetic data that uses sparse measurements to produce dense depth maps, enabling real-time deployment in unseen field robotics environments.

Contribution

The authors develop a synthetic dataset generation pipeline and a depth completion model tailored for unstructured environments, addressing scale ambiguity and data scarcity issues.

Findings

Achieves 53 ms per frame latency on Nvidia Jetson AGX Orin.

Demonstrates competitive performance across diverse real-world scenarios.

Enables real-time dense depth prediction using sparse measurements.

Abstract

Autonomous field robots operating in unstructured environments require robust perception to ensure safe and reliable operations. Recent advances in monocular depth estimation have demonstrated the potential of low-cost cameras as depth sensors; however, their adoption in field robotics remains limited due to the absence of reliable scale cues, ambiguous or low-texture conditions, and the scarcity of large-scale datasets. To address these challenges, we propose a depth completion model that trains on synthetic data and uses extremely sparse measurements from depth sensors to predict dense metric depth in unseen field robotics environments. A synthetic dataset generation pipeline tailored to field robotics enables the creation of multiple realistic datasets for training purposes. This dataset generation approach utilizes textured 3D meshes from Structure from Motion and photorealistic rendering with novel viewpoint synthesis to simulate diverse field robotics scenarios. Our approach achieves an end-to-end latency of 53 ms per frame on a Nvidia Jetson AGX Orin, enabling real-time deployment on embedded platforms. Extensive evaluation demonstrates competitive performance across diverse real-world field robotics scenarios.