Asynchronously Trained Distributed Topographic Maps

TL;DR

This paper introduces a scalable, asynchronous distributed algorithm for topographic maps that enables autonomous units to generate feature maps without centralized control, maintaining quality and efficiency.

Contribution

It presents a novel asynchronous distributed training algorithm for topographic maps using autonomous units with sparse interactions and cascade-driven updates.

Findings

Algorithm scales linearly with system size N.

Performs comparably to existing methods in classification tasks.

Empirical analysis shows robustness and efficiency of the approach.

Abstract

Topographic feature maps are low dimensional representations of data, that preserve spatial dependencies. Current methods of training such maps (e.g. self organizing maps - SOM, generative topographic maps) require centralized control and synchronous execution, which restricts scalability. We present an algorithm that uses $N$ autonomous units to generate a feature map by distributed asynchronous training. Unit autonomy is achieved by sparse interaction in time \& space through the combination of a distributed heuristic search, and a cascade-driven weight updating scheme governed by two rules: a unit i) adapts when it receives either a sample, or the weight vector of a neighbor, and ii) broadcasts its weight vector to its neighbors after adapting for a predefined number of times. Thus, a vector update can trigger an avalanche of adaptation. We map avalanching to a statistical mechanics model, which allows us to parametrize the statistical properties of cascading. Using MNIST, we empirically investigate the effect of the heuristic search accuracy and the cascade parameters on map quality. We also provide empirical evidence that algorithm complexity scales at most linearly with system size $N$. The proposed approach is found to perform comparably with similar methods in classification tasks across multiple datasets.