Stochastic Vector Quantisers

TL;DR

This paper introduces a stochastic extension to traditional vector quantization, where encoding involves sampling from a probability distribution, leading to self-organizing codebooks that can automate high-dimensional input splitting.

Contribution

It presents a novel stochastic VQ method that self-organizes and automates input vector partitioning, improving upon deterministic approaches.

Findings

Self-organization of codebooks observed in simulations

Automated splitting of high-dimensional vectors demonstrated

Enhanced encoding efficiency through stochastic sampling

Abstract

In this paper a stochastic generalisation of the standard Linde-Buzo-Gray (LBG) approach to vector quantiser (VQ) design is presented, in which the encoder is implemented as the sampling of a vector of code indices from a probability distribution derived from the input vector, and the decoder is implemented as a superposition of reconstruction vectors, and the stochastic VQ is optimised using a minimum mean Euclidean reconstruction distortion criterion, as in the LBG case. Numerical simulations are used to demonstrate how this leads to self-organisation of the stochastic VQ, where different stochastically sampled code indices become associated with different input subspaces. This property may be used to automate the process of splitting high-dimensional input vectors into low-dimensional blocks before encoding them.