Efficient Video Compression via Content-Adaptive Super-Resolution

TL;DR

This paper introduces SRVC, a content-adaptive super-resolution method that enhances existing video codecs by encoding a lightweight neural network, significantly reducing bitrates while maintaining high video quality and real-time performance.

Contribution

The paper proposes a novel hybrid video compression approach combining traditional codecs with a lightweight, content-adaptive super-resolution neural network for improved efficiency.

Findings

Achieves comparable PSNR to H.265 using only 16% of its bits-per-pixel.

Uses just 2% of DVC's bits-per-pixel to reach similar quality.

Runs at 90 frames per second on a NVIDIA V100 GPU.

Abstract

Video compression is a critical component of Internet video delivery. Recent work has shown that deep learning techniques can rival or outperform human-designed algorithms, but these methods are significantly less compute and power-efficient than existing codecs. This paper presents a new approach that augments existing codecs with a small, content-adaptive super-resolution model that significantly boosts video quality. Our method, SRVC, encodes video into two bitstreams: (i) a content stream, produced by compressing downsampled low-resolution video with the existing codec, (ii) a model stream, which encodes periodic updates to a lightweight super-resolution neural network customized for short segments of the video. SRVC decodes the video by passing the decompressed low-resolution video frames through the (time-varying) super-resolution model to reconstruct high-resolution video frames. Our results show that to achieve the same PSNR, SRVC requires 16% of the bits-per-pixel of H.265 in slow mode, and 2% of the bits-per-pixel of DVC, a recent deep learning-based video compression scheme. SRVC runs at 90 frames per second on a NVIDIA V100 GPU.