Secure AI-Driven Super-Resolution for Real-Time Mixed Reality Applications

TL;DR

This paper presents a secure, real-time super-resolution system for immersive AR/VR streaming that reduces bandwidth and latency by combining downsampling, partial encryption, and ML-based upscaling.

Contribution

It introduces a novel system that integrates content downsampling, partial encryption, and ML-driven super-resolution for efficient, secure AR/VR content delivery.

Findings

Bandwidth and latency are nearly linearly reduced with lower downsampling resolutions.

The super-resolution model accurately reconstructs full-resolution point clouds.

The system maintains low encryption/decryption overhead and inference time.

Abstract

Immersive formats such as 360{\deg} and 6DoF point cloud videos require high bandwidth and low latency, posing challenges for real-time AR/VR streaming. This work focuses on reducing bandwidth consumption and encryption/decryption delay, two key contributors to overall latency. We design a system that downsamples point cloud content at the origin server and applies partial encryption. At the client, the content is decrypted and upscaled using an ML-based super-resolution model. Our evaluation demonstrates a nearly linear reduction in bandwidth/latency, and encryption/decryption overhead with lower downsampling resolutions, while the super-resolution model effectively reconstructs the original full-resolution point clouds with minimal error and modest inference time.