Entropy Coding for Non-Rectangular Transform Blocks using Partitioned DCT Dictionaries for AV1

TL;DR

This paper proposes an entropy coding method for non-rectangular transform blocks in AV1, enabling efficient compression of signals with smooth boundary blending by modeling their sparse DCT-based representations.

Contribution

It introduces a novel entropy coding scheme tailored for partitioned DCT dictionaries to better encode non-rectangular transform coefficients in video compression.

Findings

Significant theoretical rate savings estimated for dissimilar signals.

Effective modeling of sparse DCT-based coefficients improves compression.

Method requires minimal changes at the decoder.

Abstract

Recent video codecs such as VVC and AV1 apply a Non-rectangular (NR) partitioning to combine prediction signals using a smooth blending around the boundary, followed by a rectangular transform on the whole block. The NR signal transformation is not yet supported. A transformation technique that applies the same partitioning to the 2D Discrete Cosine Transform (DCT) bases and finds a sparse representation of the NR signal in such a dictionary showed promising gains in an experimental setup outside the reference software. This method uses the regular inverse transformation at the decoder to reconstruct a rectangular signal and discards the signal outside the region of interest. This design is appealing due to the minimal changes required at the decoder. However, current entropy coding schemes are not well-suited for optimally encoding these coefficients because they are primarily designed for DCT coefficients. This work introduces an entropy coding method that efficiently codes these transform coefficients by effectively modeling their properties. The design offers significant theoretical rate savings, estimated using conditional entropy, particularly for scenarios that are more dissimilar to DCT in an experimental setup.