Radiance Fields in XR: A Survey on How Radiance Fields are Envisioned and Addressed for XR Research

TL;DR

This survey reviews how radiance fields are envisioned, implemented, and researched for XR, highlighting the current state, gaps, and future directions in integrating RF technology into XR applications.

Contribution

It systematically analyzes 365 RF papers related to XR, providing a comprehensive overview and positioning XR-specific RF research within the broader RF field.

Findings

RF research in XR is growing but remains sparse.

Detailed analysis of 66 papers reveals key implementation aspects.

The survey identifies research gaps and future opportunities.

Abstract

The development of radiance fields (RF), such as 3D Gaussian Splatting (3DGS) and Neural Radiance Fields (NeRF), has revolutionized interactive photorealistic view synthesis and presents enormous opportunities for XR research and applications. However, despite the exponential growth of RF research, RF-related contributions to the XR community remain sparse. To better understand this research gap, we performed a systematic survey of current RF literature to analyze (i) how RF is envisioned for XR applications, (ii) how they have already been implemented, and (iii) the remaining research gaps. We collected 365 RF contributions related to XR from computer vision, computer graphics, robotics, multimedia, human-computer interaction, and XR communities, seeking to answer the above research questions. Among the 365 papers, we performed an analysis of 66 papers that already addressed a detailed aspect of RF research for XR. With this survey, we extended and positioned XR-specific RF research topics in the broader RF research field and provide a helpful resource for the XR community to navigate within the rapid development of RF research.