The Canonical Distortion Measure for Vector Quantization and Function Approximation

TL;DR

This paper introduces a canonical distortion measure (CDM) for evaluating vector quantization quality, which is better suited for natural signals like speech and images, and demonstrates its effectiveness through theoretical analysis and neural network training.

Contribution

It defines a new canonical distortion measure (CDM) induced by function environments, providing a more appropriate metric for natural signals and offering algorithms for neural network implementation.

Findings

CDM can be computed in closed form for various function classes.

Optimizing reconstruction error with CDM yields optimal piecewise constant approximations.

Neural network training using CDM shows promising experimental results.

Abstract

To measure the quality of a set of vector quantization points a means of measuring the distance between a random point and its quantization is required. Common metrics such as the {\em Hamming} and {\em Euclidean} metrics, while mathematically simple, are inappropriate for comparing natural signals such as speech or images. In this paper it is shown how an {\em environment} of functions on an input space $X$ induces a {\em canonical distortion measure} (CDM) on X. The depiction 'canonical" is justified because it is shown that optimizing the reconstruction error of X with respect to the CDM gives rise to optimal piecewise constant approximations of the functions in the environment. The CDM is calculated in closed form for several different function classes. An algorithm for training neural networks to implement the CDM is presented along with some encouraging experimental results.