Interacting with New York City Data by HoloLens through Remote Rendering

TL;DR

This paper presents a remote rendering XR system for HoloLens that enhances user experience when interacting with large 3D city data by offloading rendering to a remote server, significantly improving QoE.

Contribution

It introduces a novel remote rendering approach for XR devices handling large datasets, demonstrating improved performance over local rendering in a real-world city model.

Findings

Remote rendering improves QoE by at least 21%.

Network traffic pattern analysis clarifies data flow in the system.

System effectively handles large-scale 3D city data on HoloLens.

Abstract

In the digital era, Extended Reality (XR) is considered the next frontier. However, XR systems are computationally intensive, and they must be implemented within strict latency constraints. Thus, XR devices with finite computing resources are limited in terms of quality of experience (QoE) they can offer, particularly in cases of big 3D data. This problem can be effectively addressed by offloading the highly intensive rendering tasks to a remote server. Therefore, we proposed a remote rendering enabled XR system that presents the 3D city model of New York City on the Microsoft HoloLens. Experimental results indicate that remote rendering outperforms local rendering for the New York City model with significant improvement in average QoE by at least 21%. Additionally, we clarified the network traffic pattern in the proposed XR system developed under the OpenXR standard.