RaCalNet: Radar Calibration Network for Sparse-Supervised Metric Depth Estimation

TL;DR

RaCalNet introduces a radar calibration framework that learns accurate depth estimation from sparse radar supervision, reducing reliance on dense LiDAR data and achieving high-quality results with minimal supervision.

Contribution

The paper presents RaCalNet, a novel sparse-supervised radar calibration network that produces accurate depth maps without dense supervision, improving efficiency and reducing data requirements.

Findings

Achieves comparable accuracy to dense-supervised methods on benchmark datasets.

Reduces RMSE by 34.89% in real-world deployment scenarios.

Produces depth maps with clear object contours and fine textures.

Abstract

Dense depth estimation using millimeter-wave radar typically requires dense LiDAR supervision, generated via multi-frame projection and interpolation, for guiding the learning of accurate depth from sparse radar measurements and RGB images. However, this paradigm is both costly and data-intensive. To address this, we propose RaCalNet, a novel framework that eliminates the need for dense supervision by using sparse LiDAR to supervise the learning of refined radar measurements, resulting in a supervision density of merely around 1\% compared to dense-supervised methods. RaCalNet is composed of two key modules. The Radar Recalibration module performs radar point screening and pixel-wise displacement refinement, producing accurate and reliable depth priors from sparse radar inputs. These priors are then used by the Metric Depth Optimization module, which learns to infer scene-level scale priors and fuses them with monocular depth predictions to achieve metrically accurate outputs. This modular design enhances structural consistency and preserves fine-grained geometric details. Despite relying solely on sparse supervision, RaCalNet produces depth maps with clear object contours and fine-grained textures, demonstrating superior visual quality compared to state-of-the-art dense-supervised methods. Quantitatively, it achieves performance comparable to existing methods on the ZJU-4DRadarCam dataset and yields a 34.89\% RMSE reduction in real-world deployment scenarios. We plan to gradually release the code and models in the future at https://github.com/818slam/RaCalNet.git.