A Hybrid mmWave and Camera System for Long-Range Depth Imaging
Akarsh Prabhakara, Diana Zhang, Chao Li, Sirajum Munir, Aswin, Sankanaryanan, Anthony Rowe, Swarun Kumar

TL;DR
This paper introduces Metamoran, a hybrid system combining mmWave radar and camera data to achieve high-resolution, long-range depth imaging in cluttered environments, surpassing existing methods in accuracy and range.
Contribution
The paper presents a novel method to convert camera-derived object shapes into RF I/Q signals, enhancing radar processing for improved long-range depth imaging.
Findings
Estimates static object depth up to 90 m and moving objects up to 305 m.
Achieves median depth error of 28 cm, 13× better than naive radar+camera baseline.
Outperforms monocular depth estimation by 23×.
Abstract
mmWave radars offer excellent depth resolution even at very long ranges owing to their high bandwidth. But their angular resolution is at least an order-of-magnitude worse than camera and lidar systems. Hence, mmWave radar is not a capable 3-D imaging solution in isolation. We propose Metamoran, a system that combines the complimentary strengths of radar and camera to obtain accurate, high resolution depth images over long ranges even in high clutter environments, all from a single fixed vantage point. Metamoran enables rich long-range depth imaging with applications in security and surveillance, roadside safety infrastructure and wide-area mapping. Our approach leverages the high angular resolution from cameras using computer vision techniques, including image segmentation and monocular depth estimation, to obtain object shape. Our core contribution is a method to convert this object…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsIndoor and Outdoor Localization Technologies · Advanced Optical Sensing Technologies · Microwave Imaging and Scattering Analysis
