Adaptive High-Frequency Preprocessing for Video Coding

TL;DR

This paper introduces an end-to-end learning framework that adaptively preprocesses high-frequency components in videos to improve visual quality while reducing bitrate, using a novel frequency-attentive prediction network.

Contribution

It proposes a novel Frequency-attentive Feature pyramid Prediction Network (FFPN) for adaptive high-frequency preprocessing in video coding, optimizing the tradeoff between quality and bitrate.

Findings

Achieves significant bitrate savings while maintaining visual quality.

Demonstrates superior performance on multiple datasets.

Enhances subjective video quality effectively.

Abstract

High-frequency components are crucial for maintaining video clarity and realism, but they also significantly impact coding bitrate, resulting in increased bandwidth and storage costs. This paper presents an end-to-end learning-based framework for adaptive high-frequency preprocessing to enhance subjective quality and save bitrate in video coding. The framework employs the Frequency-attentive Feature pyramid Prediction Network (FFPN) to predict the optimal high-frequency preprocessing strategy, guiding subsequent filtering operators to achieve the optimal tradeoff between bitrate and quality after compression. For training FFPN, we pseudo-label each training video with the optimal strategy, determined by comparing the rate-distortion (RD) performance across different preprocessing types and strengths. Distortion is measured using the latest quality assessment metric. Comprehensive evaluations on multiple datasets demonstrate the visually appealing enhancement capabilities and bitrate savings achieved by our framework.