Concerning the differentiability of the energy function in vector quantization algorithms

TL;DR

This paper investigates the conditions under which the energy function in vector quantization algorithms, like K-means and Neural-Gas, is differentiable or a potential, revealing it is often a pseudo-potential rather than a true potential.

Contribution

It introduces the concept of pseudo-potentials for energy functions in vector quantization, expanding understanding of convergence properties for these algorithms.

Findings

K-means energy function is a pseudo-potential, not a potential.

Neural-Gas energy function is generally not a potential.

Many neural network-based vector quantization algorithms develop pseudo-potentials.

Abstract

The adaptation rule for Vector Quantization algorithms, and consequently the convergence of the generated sequence, depends on the existence and properties of a function called the energy function, defined on a topological manifold. Our aim is to investigate the conditions of existence of such a function for a class of algorithms examplified by the initial ''K-means'' and Kohonen algorithms. The results presented here supplement previous studies and show that the energy function is not always a potential but at least the uniform limit of a series of potential functions which we call a pseudo-potential. Our work also shows that a large number of existing vector quantization algorithms developped by the Artificial Neural Networks community fall into this category. The framework we define opens the way to study the convergence of all the corresponding adaptation rules at once, and a theorem gives promising insights in that direction. We also demonstrate that the ''K-means'' energy function is a pseudo-potential but not a potential in general. Consequently, the energy function associated to the ''Neural-Gas'' is not a potential in general.