Orthonormal Matrix Codebook Design for Adaptive Transform Coding

TL;DR

This paper introduces a new algorithm for designing orthonormal transform matrix codebooks that optimize adaptive transform coding for non-stationary processes, improving coding efficiency.

Contribution

It presents a block-coordinate descent algorithm that efficiently finds orthonormal transform matrices by minimizing mean square error, applicable to various adaptive coding scenarios.

Findings

Optimized codebooks outperform standard DCT in video coding residuals.

Algorithm effectively handles matrix-orthonormality constraints on the Stiefel manifold.

Preliminary results show improved coding performance with adaptive transform codes.

Abstract

A novel algorithm for designing optimized orthonormal transform-matrix codebooks for adaptive transform coding of a non-stationary vector process is proposed. This algorithm relies on a block-wise stationary model of a non-stationary process and finds a codebook of transform-matrices by minimizing the end-to-end mean square error of transform coding averaged over the distribution of stationary blocks of vectors. The algorithm, which belongs to the class of block-coordinate descent algorithms, solves an intermediate minimization problem involving matrix-orthonormality constraints in a computationally efficient manner by mapping the problem from the Euclidean space to the Stiefel manifold. As such, the algorithm can be broadly applied to any adaptive transform coding problem. Preliminary results obtained with inter-prediction residuals in an H265 video codec are presented to demonstrate the advantage of optimized adaptive transform codes over non-adaptive codes based on the standard DCT.