Federated Deep Reinforcement Learning-based Bitrate Adaptation for Dynamic Adaptive Streaming over HTTP

TL;DR

This paper introduces a federated deep reinforcement learning approach for adaptive video streaming that trains models locally on clients, improving bitrate adaptation across diverse network environments without central data collection.

Contribution

It presents a novel federated learning framework integrated with DRL for bitrate adaptation, enabling environment-specific training without sharing raw data.

Findings

FDRLABR outperforms traditional methods in diverse network conditions.

Different DRL algorithms within FDRLABR show improved QoE.

Federated approach preserves privacy while enhancing adaptation quality.

Abstract

In video streaming over HTTP, the bitrate adaptation selects the quality of video chunks depending on the current network condition. Some previous works have applied deep reinforcement learning (DRL) algorithms to determine the chunk's bitrate from the observed states to maximize the quality-of-experience (QoE). However, to build an intelligent model that can predict in various environments, such as 3G, 4G, Wifi, \textit{etc.}, the states observed from these environments must be sent to a server for training centrally. In this work, we integrate federated learning (FL) to DRL-based rate adaptation to train a model appropriate for different environments. The clients in the proposed framework train their model locally and only update the weights to the server. The simulations show that our federated DRL-based rate adaptations, called FDRLABR with different DRL algorithms, such as deep Q-learning, advantage actor-critic, and proximal policy optimization, yield better performance than the traditional bitrate adaptation methods in various environments.