Conditional Residual Coding: A Remedy for Bottleneck Problems in Conditional Inter Frame Coding

TL;DR

This paper introduces conditional residual coding, a novel approach that mitigates information bottlenecks in neural network-based video coding, combining the advantages of residual and conditional coding for improved performance.

Contribution

It proposes the conditional residual coding concept, derived from information theory, to address bottleneck issues in conditional neural video coding.

Findings

Reduces bottleneck effects in conditional coding

Maintains theoretical performance of conditional coders

Alleviates disadvantages of conditional coders while preserving their benefits

Abstract

Conditional coding is a new video coding paradigm enabled by neural-network-based compression. It can be shown that conditional coding is in theory better than the traditional residual coding, which is widely used in video compression standards like HEVC or VVC. However, on closer inspection, it becomes clear that conditional coders can suffer from information bottlenecks in the prediction path, i.e., that due to the data processing inequality not all information from the prediction signal can be passed to the reconstructed signal, thereby impairing the coder performance. In this paper we propose the conditional residual coding concept, which we derive from information theoretical properties of the conditional coder. This coder significantly reduces the influence of bottlenecks, while maintaining the theoretical performance of the conditional coder. We provide a theoretical analysis of the coding paradigm and demonstrate the performance of the conditional residual coder in a practical example. We show that conditional residual coders alleviate the disadvantages of conditional coders while being able to maintain their advantages over residual coders. In the spectrum of residual and conditional coding, we can therefore consider them as ``the best from both worlds''.