Advances In Video Compression System Using Deep Neural Network: A Review And Case Studies

TL;DR

This paper reviews recent advances in video compression using deep neural networks, highlighting innovative approaches in pre-processing, coding, and post-processing, supported by three detailed case studies demonstrating improved efficiency and quality.

Contribution

It provides a comprehensive review of DNN-based video compression techniques and presents three novel case studies showcasing practical implementations and benefits.

Findings

DNN-based scene understanding improves video coding efficiency.

End-to-end neural coding achieves compact, data-driven video compression.

Neural adaptive filters enhance compressed frame quality.

Abstract

Significant advances in video compression system have been made in the past several decades to satisfy the nearly exponential growth of Internet-scale video traffic. From the application perspective, we have identified three major functional blocks including pre-processing, coding, and post-processing, that have been continuously investigated to maximize the end-user quality of experience (QoE) under a limited bit rate budget. Recently, artificial intelligence (AI) powered techniques have shown great potential to further increase the efficiency of the aforementioned functional blocks, both individually and jointly. In this article, we review extensively recent technical advances in video compression system, with an emphasis on deep neural network (DNN)-based approaches; and then present three comprehensive case studies. On pre-processing, we show a switchable texture-based video coding example that leverages DNN-based scene understanding to extract semantic areas for the improvement of subsequent video coder. On coding, we present an end-to-end neural video coding framework that takes advantage of the stacked DNNs to efficiently and compactly code input raw videos via fully data-driven learning. On post-processing, we demonstrate two neural adaptive filters to respectively facilitate the in-loop and post filtering for the enhancement of compressed frames. Finally, a companion website hosting the contents developed in this work can be accessed publicly at https://purdueviper.github.io/dnn-coding/.