Foveated Compression for Immersive Telepresence Visualization

TL;DR

This paper introduces a foveated compression method for immersive telepresence videos that leverages eye tracking to reduce bandwidth by transmitting high-fidelity foveal regions and lower quality peripheral areas, maintaining immersion.

Contribution

It presents a novel adaptive compression technique that adjusts video quality based on eye tracking data, compatible with existing codecs, to optimize bandwidth in immersive telepresence.

Findings

Bandwidth reduced to one-third without losing immersion

Effective integration with existing video codecs

Qualitative and quantitative validation of fidelity

Abstract

Immersive televisualization is important both for telepresence and teleoperation, but resolution and fidelity are often limited by communication bandwidth constraints. We propose a lightweight method for foveated compression of immersive televisualization video streams that can be easily integrated with common video codecs, reducing the required bandwidth if eye tracking data is available. Specifically, we show how to spatially adjust the Quantization Parameter of modern block-based video codecs in a adaptive way based on eye tracking information. The foveal region is transmitted with high fidelity while quality is reduced in the peripheral region, saving bandwidth. We integrate our method with the NimbRo avatar system, which won the ANA Avatar XPRIZE competition. Our experiments show that bandwidth can be reduced to a third without sacrificing immersion. We analyze transmission fidelity with qualitative examples and report quantitative results.