Joint Optimization of Primary and Secondary Transforms Using Rate-Distortion Optimized Transform Design

TL;DR

This paper introduces a joint optimization framework for primary and secondary transforms in video coding, improving rate-distortion performance by data-driven design and clustering, especially for AVM residual data.

Contribution

It proposes a novel joint design method for primary and secondary transforms using rate-distortion optimized clustering, enhancing coding efficiency over traditional methods.

Findings

Joint clustering reduces total RD cost.

Jointly optimized transforms outperform traditional KLTs.

Improved coding efficiency demonstrated with AVM residual data.

Abstract

Data-dependent transforms are increasingly being incorporated into next-generation video coding systems such as AVM, a codec under development by the Alliance for Open Media (AOM), and VVC. To circumvent the computational complexities associated with implementing non-separable data-dependent transforms, combinations of separable primary transforms and non-separable secondary transforms have been studied and integrated into video coding standards. These codecs often utilize rate-distortion optimized transforms (RDOT) to ensure that the new transforms complement existing transforms like the DCT and the ADST. In this work, we propose an optimization framework for jointly designing primary and secondary transforms from data through a rate-distortion optimized clustering. Primary transforms are assumed to follow a path-graph model, while secondary transforms are non-separable. We empirically evaluate our proposed approach using AVM residual data and demonstrate that 1) the joint clustering method achieves lower total RD cost in the RDOT design framework, and 2) jointly optimized separable path-graph transforms (SPGT) provide better coding efficiency compared to separable KLTs obtained from the same data.