Imitation Learning for Adaptive Video Streaming with Future Adversarial Information Bottleneck Principle

TL;DR

This paper introduces a novel imitation learning approach with an adversarial information bottleneck for adaptive video streaming, significantly improving QoE by leveraging offline optimal scenarios and mitigating future information leakage.

Contribution

It combines imitation learning with an information bottleneck and adversarial framework to enhance RL-based ABR algorithms, addressing overfitting and performance fluctuations.

Findings

Achieved 7.30% average QoE improvement.

Reduced average ranking by 30.01%.

Effectively mitigated future information leakage.

Abstract

Adaptive video streaming plays a crucial role in ensuring high-quality video streaming services. Despite extensive research efforts devoted to Adaptive BitRate (ABR) techniques, the current reinforcement learning (RL)-based ABR algorithms may benefit the average Quality of Experience (QoE) but suffers from fluctuating performance in individual video sessions. In this paper, we present a novel approach that combines imitation learning with the information bottleneck technique, to learn from the complex offline optimal scenario rather than inefficient exploration. In particular, we leverage the deterministic offline bitrate optimization problem with the future throughput realization as the expert and formulate it as a mixed-integer non-linear programming (MINLP) problem. To enable large-scale training for improved performance, we propose an alternative optimization algorithm that efficiently solves the MINLP problem. To address the issues of overfitting due to the future information leakage in MINLP, we incorporate an adversarial information bottleneck framework. By compressing the video streaming state into a latent space, we retain only action-relevant information. Additionally, we introduce a future adversarial term to mitigate the influence of future information leakage, where Model Prediction Control (MPC) policy without any future information is employed as the adverse expert. Experimental results demonstrate the effectiveness of our proposed approach in significantly enhancing the quality of adaptive video streaming, providing a 7.30\% average QoE improvement and a 30.01\% average ranking reduction.