From Ember to Blaze: Swift Interactive Video Adaptation via Meta-Reinforcement Learning

TL;DR

This paper introduces Fiammetta, a meta-reinforcement learning algorithm for interactive video bitrate adaptation that leverages short-term network continuity for rapid online adjustment, significantly enhancing quality of experience.

Contribution

Fiammetta is the first meta-RL-based adaptive streaming algorithm that uses offline meta-training and real-time probing for fast adaptation to network fluctuations.

Findings

Fiammetta improves video bitrate by up to 16.2%.

Fiammetta maintains low stalling rates.

Short-term network continuity enables effective few-shot learning.

Abstract

Maximizing quality of experience (QoE) for interactive video streaming has been a long-standing challenge, as its delay-sensitive nature makes it more vulnerable to bandwidth fluctuations. While reinforcement learning (RL) has demonstrated great potential, existing works are either limited by fixed models or require enormous data/time for online adaptation, which struggle to fit time-varying and diverse network states. Driven by these practical concerns, we perform large-scale measurements on WeChat for Business's interactive video service to study real-world network fluctuations. Surprisingly, our analysis shows that, compared to time-varying network metrics, network sequences exhibit noticeable short-term continuity, sufficient for few-shot learning requirements. We thus propose Fiammetta, the first meta-RL-based bitrate adaptation algorithm for interactive video streaming. Building on the short-term continuity, Fiammetta accumulates learning experiences through offline meta-training and enables fast online adaptation to changing network states through a few gradient updates. Moreover, Fiammetta innovatively incorporates a probing mechanism for real-time monitoring of network states, and proposes an adaptive meta-testing mechanism for seamless adaptation. We implement Fiammetta on a testbed whose end-to-end network follows the real-world WeChat for Business traces. The results show that Fiammetta outperforms prior algorithms significantly, improving video bitrate by 3.6%-16.2% without increasing stalling rate.