Quality-Aware Deep Reinforcement Learning for Streaming in Infrastructure-Assisted Connected Vehicles

TL;DR

This paper introduces a deep reinforcement learning-based video streaming scheme for infrastructure-assisted vehicular networks, optimizing content delivery, quality, and proactive chunk pushing to enhance streaming quality and efficiency in high-mobility scenarios.

Contribution

It proposes a novel DRL-based dynamic video delivery scheme using DDPG to adaptively manage content quality and proactive delivery in mmWave vehicular networks.

Findings

DRL agent learns policies that improve streaming quality

Reduces packet drops and playback stalls

Saves backhaul bandwidth

Abstract

This paper proposes a deep reinforcement learning-based video streaming scheme for mobility-aware vehicular networks, e.g., vehicles on the highway. We consider infrastructure-assisted and mmWave-based scenarios in which the macro base station (MBS) cannot directly provide the streaming service to vehicles due to the short range of mmWave beams so that small mmWave base stations (mBSs) along the road deliver the desired videos to users. For a smoother streaming service, the MBS proactively pushes video chunks to mBSs. This is done to support vehicles that are currently covered and/or will be by each mBS. We formulate the dynamic video delivery scheme that adaptively determines 1) which content, 2) what quality and 3) how many chunks to be proactively delivered from the MBS to mBSs using Markov decision process (MDP). Since it is difficult for the MBS to track all the channel conditions and the network states have extensive dimensions, we adopt the deep deterministic policy gradient (DDPG) algorithm for the DRL-based video delivery scheme. This paper finally shows that the DRL agent learns a streaming policy that pursues high average quality while limiting packet drops, avoiding playback stalls, reducing quality fluctuations and saving backhaul usage.