Variable Rate Learned Wavelet Video Coding using Temporal Layer Adaptivity

TL;DR

This paper introduces a variable rate learned wavelet video coding framework with temporal layer adaptivity, achieving significant bitrate savings and improved coding efficiency through multi-stage training and quality adaptation.

Contribution

It presents a novel variable rate support mechanism and a multi-stage training strategy for learned wavelet video coders, enhancing scalability and efficiency.

Findings

Achieves at least -32% bitrate savings over baseline models.

Demonstrates effective quality adaptation across temporal layers.

Provides open-source training and inference code.

Abstract

Learned wavelet video coders provide an explainable framework by performing discrete wavelet transforms in temporal, horizontal, and vertical dimensions. With a temporal transform based on motion-compensated temporal filtering (MCTF), spatial and temporal scalability is obtained. In this paper, we introduce variable rate support and a mechanism for quality adaption to different temporal layers for a higher coding efficiency. Moreover, we propose a multi-stage training strategy that allows training with multiple temporal layers. Our experiments demonstrate Bj{\o}ntegaard Delta bitrate savings of at least -32% compared to a learned MCTF model without these extensions. Training and inference code is available at: https://github.com/FAU-LMS/Learned-pMCTF.