Enhancement or Super-Resolution: Learning-based Adaptive Video Streaming with Client-Side Video Processing

TL;DR

This paper introduces ENAVS, a deep reinforcement learning framework for adaptive video streaming that leverages client-side enhancement to improve quality, outperforming super-resolution methods and increasing user experience by up to 14%.

Contribution

The paper presents a novel DRL-based joint bitrate adaptation and enhancement algorithm that optimizes video quality in fluctuating network and device conditions.

Findings

ENAVS delivers 5%-14% higher QoE compared to traditional methods.

Client-side enhancement outperforms super-resolution in video quality metrics.

The approach is effective under real-world bandwidth and device constraints.

Abstract

The rapid development of multimedia and communication technology has resulted in an urgent need for high-quality video streaming. However, robust video streaming under fluctuating network conditions and heterogeneous client computing capabilities remains a challenge. In this paper, we consider an enhancement-enabled video streaming network under a time-varying wireless network and limited computation capacity. "Enhancement" means that the client can improve the quality of the downloaded video segments via image processing modules. We aim to design a joint bitrate adaptation and client-side enhancement algorithm toward maximizing the quality of experience (QoE). We formulate the problem as a Markov decision process (MDP) and propose a deep reinforcement learning (DRL)-based framework, named ENAVS. As video streaming quality is mainly affected by video compression, we demonstrate that the video enhancement algorithm outperforms the super-resolution algorithm in terms of signal-to-noise ratio and frames per second, suggesting a better solution for client processing in video streaming. Ultimately, we implement ENAVS and demonstrate extensive testbed results under real-world bandwidth traces and videos. The simulation shows that ENAVS is capable of delivering 5%-14% more QoE under the same bandwidth and computing power conditions as conventional ABR streaming.