A Discussion on Parallelization Schemes for Stochastic Vector Quantization Algorithms

TL;DR

This paper explores parallelization strategies for stochastic Vector Quantization algorithms, demonstrating that a new scheme can achieve significant speed-ups on distributed systems, including cloud platforms like Microsoft Azure.

Contribution

Introduces a novel parallelization scheme for stochastic Vector Quantization that outperforms intuitive methods and adapts well to slow communication environments.

Findings

The intuitive parallelization scheme does not improve performance.

A new distributed scheme achieves expected speed-ups.

Speed-ups up to 32 Virtual Machines on Azure.

Abstract

This paper studies parallelization schemes for stochastic Vector Quantization algorithms in order to obtain time speed-ups using distributed resources. We show that the most intuitive parallelization scheme does not lead to better performances than the sequential algorithm. Another distributed scheme is therefore introduced which obtains the expected speed-ups. Then, it is improved to fit implementation on distributed architectures where communications are slow and inter-machines synchronization too costly. The schemes are tested with simulated distributed architectures and, for the last one, with Microsoft Windows Azure platform obtaining speed-ups up to 32 Virtual Machines.