DHR+S: Distributed Hybrid Rendering with Realistic Real-time Shadows for Interactive Thin Client Metaverse and Game Applications

TL;DR

This paper introduces a distributed hybrid rendering system that combines cloud ray tracing with local rasterization to deliver high-fidelity, real-time shadows for interactive metaverse and game applications on low-capability devices, even under poor network conditions.

Contribution

It presents a novel hybrid rendering approach that enhances shadow realism with soft edges and ambient occlusion approximation, while reducing bandwidth and tolerating network delays.

Findings

Achieves realistic soft shadows with low bandwidth usage.

Maintains interactive frame rates on low-capability devices.

Tolerates network delays up to 200 ms with minimal shadow distortion.

Abstract

Distributed hybrid rendering (DHR) is a real-time rendering approach that incorporates cloud-based ray tracing with locally rasterized graphics for interactive thin client metaverse and game applications. With cloud assistance, DHR can generate high-fidelity ray-traced graphics contents remotely and deliver them to thin clients with low graphics capability, including standalone extended reality devices and mobile phones, while maintaining interactive frame rates for users under adverse network conditions. DHR can already achieve the effect of ray-traced hard shadows that form with the occlusion of direct illumination. We enhance the realism of these shadows by softening their edges with the direction of rays traced and approximating the occlusion of indirect illumination by reconstructing ray-traced ambient occlusion with a modified version of spatiotemporal variance-guided filtering. Our technique uses only 20-30% of the bandwidth of remote rendering and is also tolerant of delays of up to 200 ms with only slight distortion to the shadows along object edges.