Millimeter Wave MIMO based Depth Maps for Wireless Virtual and Augmented Reality
Abdelrahman Taha, Qi Qu, Sam Alex, Ping Wang, William L. Abbott, and, Ahmed Alkhateeb

TL;DR
This paper introduces a novel method for constructing high-resolution depth maps for AR/VR using mmWave MIMO communication systems, overcoming limitations of traditional RGB-D cameras and leveraging mmWave signal properties.
Contribution
It proposes a new framework with specialized beamforming codebooks and joint beam processing for accurate depth mapping using practical mmWave antenna arrays.
Findings
Simulation results demonstrate high accuracy of the proposed depth maps.
mmWave based depth perception outperforms RGB-based methods in key scenarios.
The approach is feasible with current mmWave technology.
Abstract
Augmented and virtual reality systems (AR/VR) are rapidly becoming key components of the wireless landscape. For immersive AR/VR experience, these devices should be able to construct accurate depth perception of the surrounding environment. Current AR/VR devices rely heavily on using RGB-D depth cameras to achieve this goal. The performance of these depth cameras, however, has clear limitations in several scenarios, such as the cases with shiny objects, dark surfaces, and abrupt color transition among other limitations. In this paper, we propose a novel solution for AR/VR depth map construction using mmWave MIMO communication transceivers. This is motivated by the deployment of advanced mmWave communication systems in future AR/VR devices for meeting the high data rate demands and by the interesting propagation characteristics of mmWave signals. Accounting for the constraints on these…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
