Causal-Aware Intelligent QoE Optimization for VR Interaction with Adaptive Keyframe Extraction

TL;DR

This paper introduces a causal-aware reinforcement learning framework for optimizing VR QoE by adaptively managing keyframes, bandwidth, and computational resources, leading to reduced latency and improved user experience.

Contribution

It presents a novel causal inference-based RL method, PS-CDDPG, for joint optimization of VR parameters, integrating a new QoE metric and addressing fairness constraints.

Findings

Significantly reduces interactive latency in VR.

Enhances QoE compared to benchmark methods.

Maintains fairness in resource allocation.

Abstract

The optimization of quality of experience (QoE) in multi-user virtual reality (VR) interactions demands a delicate balance between ultra-low latency, high-fidelity motion synchronization, and equitable resource allocation. While adaptive keyframe extraction mitigates transmission overhead, existing approaches often overlook the causal relationships among allocated bandwidth, CPU frequency, and user perception, limiting QoE gains. This paper proposes an intelligent framework to maximize QoE by integrating adaptive keyframe extraction with causal-aware reinforcement learning (RL). First, a novel QoE metric is formulated using the Weber-Fechner Law, combining perceptual sensitivity, attention-driven priorities, and motion reconstruction accuracy. The QoE optimization problem is then modeled as a mixed integer programming (MIP) task, jointly optimizing keyframe ratios, bandwidth, and computational resources under horizon-fairness constraints. We propose Partial State Causal Deep Deterministic Policy Gradient (PS-CDDPG), which integrates the Deep Deterministic Policy Gradient (DDPG) method with causal influence detection. By leveraging causal information regarding how QoE is influenced and determined by various actions, we explore actions guided by weights calculated from causal inference (CI), which in turn improves training efficiency. Experiments conducted with the CMU Motion Capture Database demonstrate that our framework significantly reduces interactive latency, enhances QoE, and maintains fairness, achieving superior performance compared to benchmark methods.