Remote Rendering for Virtual Reality: performance comparison of multimedia frameworks and protocols

TL;DR

This paper compares the performance of multimedia frameworks and streaming protocols for remote rendering in VR, focusing on WebRTC, DASH, and QUIC-based protocols over WIFI and 5G networks.

Contribution

It introduces an integrated testbed with GStreamer and FFmpeg for evaluating remote rendering performance using various streaming protocols in XR applications.

Findings

QUIC-based protocols show potential advantages over WebRTC and DASH.

Performance varies significantly between WIFI and 5G networks.

The testbed enables advanced research in remote XR rendering.

Abstract

The increasing complexity of Extended Reality (XR) applications demands substantial processing power and high bandwidth communications, often unavailable on lightweight devices. Remote rendering consists of offloading processing tasks to a remote node with a powerful GPU, delivering the rendered content to the end device. The delivery is usually performed through popular streaming protocols such as Web Real-Time Communications (WebRTC), offering a data channel for interactions, or Dynamic Adaptive Streaming over HTTP (DASH), better suitable for scalability. Moreover, new streaming protocols based on QUIC are emerging as potential replacements for WebRTC and DASH and offer benefits like connection migration, stream multiplexing and multipath delivery. This work describes the integration of the two most popular multimedia frameworks, GStreamer and FFmpeg, with a rendering engine acting as a Remote Renderer, and analyzes their performance when offering different protocols for delivering the rendered content to the end device over WIFI or 5G. This solution constitutes a beyond state-of-the-art testbed to conduct cutting-edge research in the XR field.