Prediction and Reference Quality Adaptation for Learned Video Compression

TL;DR

This paper introduces novel modules for learned video compression that adapt prediction and reference quality, significantly improving compression efficiency by reducing error propagation and better utilizing temporal prediction.

Contribution

It proposes a confidence-based prediction quality adaptation module and a reference quality adaptation module with a new training strategy, enhancing learned video codecs' ability to adapt to varying prediction and reference qualities.

Findings

Improved compression performance demonstrated in experiments.

Effective suppression of low-quality predictions.

Enhanced adaptability to diverse reference qualities.

Abstract

Temporal prediction is one of the most important technologies for video compression. Various prediction coding modes are designed in traditional video codecs. Traditional video codecs will adaptively to decide the optimal coding mode according to the prediction quality and reference quality. Recently, learned video codecs have made great progress. However, they did not effectively address the problem of prediction and reference quality adaptation, which limits the effective utilization of temporal prediction and reduction of reconstruction error propagation. Therefore, in this paper, we first propose a confidence-based prediction quality adaptation (PQA) module to provide explicit discrimination for the spatial and channel-wise prediction quality difference. With this module, the prediction with low quality will be suppressed and that with high quality will be enhanced. The codec can adaptively decide which spatial or channel location of predictions to use. Then, we further propose a reference quality adaptation (RQA) module and an associated repeat-long training strategy to provide dynamic spatially variant filters for diverse reference qualities. With these filters, our codec can adapt to different reference qualities, making it easier to achieve the target reconstruction quality and reduce the reconstruction error propagation. Experimental results verify that our proposed modules can effectively help our codec achieve a higher compression performance.