Error analysis of quantization combined with Hadamard transforms

TL;DR

This paper provides a theoretical analysis of error bounds and component magnitude limits in image coding processes that use Hadamard transforms and dead-zone quantization, aiding in optimizing compression and storage.

Contribution

It introduces mathematical formulas for error and magnitude bounds in Hadamard-based image coding, linking matrix properties to quantization performance.

Findings

Derived error bounds depend on matrix size and quantizer parameters.

Established bounds on maximum component magnitude for the coding process.

Connected Hadamard matrix norms to maximal excess in matrix equivalence classes.

Abstract

In this paper, we consider an image coding process consisting of the following four steps: a direct transformation, a direct quantization, an inverse quantization, and an inverse transformation, where Hadamard transforms are used for the transformation steps and a dead-zone quantizer is used for the quantization. The aim of this paper is to provide a theoretical tool for analyzing this process. We discuss error bounds for this process and bounds on the largest absolute value that the components of the result can attain. In order to obtain these bounds, we use methods of linear algebra and properties of Hadamard matrices. The obtained formulae depend on the size of the matrices, the parameters of the quantizer and the dequantizer, and a bound on the source values. Knowing the error bounds helps control the trade-off between compression efficiency and output quality. Knowing the bounds on the largest absolute value helps decide how many bits are needed to store the result. In addition, we demonstrate a connection between the norm $\|\mathbf{H}\|_{\infty, 1}$ of a Hadamard matrix $\mathbf{H}$ and the maximal excess $\sigma([\mathbf{H}])$ of the equivalence class containing $\mathbf{H}$.