ReVo: A Cross-Layer Reliable Volumetric Videoconferencing System

TL;DR

ReVo is a cross-layer, loss-resilient volumetric videoconferencing system that enhances visual quality and reduces freezes under packet loss by jointly recovering RGB and depth data with neural methods.

Contribution

It introduces a novel cross-layer, modality-aware design for volumetric video recovery, combining network-layer protection and neural reconstruction to improve robustness.

Findings

ReVo improves median SSIM by up to 32% for RGB and 13% for depth.

ReVo reduces video freezes by up to 95.7%.

It operates effectively on desktop-grade hardware over WebRTC.

Abstract

Volumetric videoconferencing enables immersive six Degrees of Freedom interactions by jointly transmitting visual appearance and 3D geometry. However, delivering volumetric video over today's networks remains challenging due to high bandwidth demands, strict real-time latency constraints, and frequent packet loss. Packet loss not only degrades visual quality but also corrupts geometric structure, leading to severe artifacts and video freezes that significantly degrade Quality of Experience. Existing solutions either optimize volumetric videos assuming reliable networks or focus on loss recovery for 2D video, and are insufficient for volumetric videoconferencing. In this paper, we present ReVo, a loss-resilient volumetric videoconferencing system that jointly recovers RGB and depth content under packet loss while meeting real-time constraints on desktop-grade hardware. ReVo leverages the insight that effective recovery requires a cross-layer, modality-aware design. It decouples volumetric video into RGB and depth streams, selectively protects critical content using network-layer FEC, and reconstructs corrupted non-critical frames using a post-decode neural recovery module. ReVo is implemented end-to-end over WebRTC and supports both traditional and neural video codecs. Our evaluations using real-world loss traces show that ReVo improves median SSIM by up to 32% (resp. 13%) for RGB (resp. depth) content and reduces video freezes by up to 95.7% compared to existing techniques.