An Implementation of Back-Propagation Learning on GF11, a Large SIMD Parallel Computer

TL;DR

This paper presents a parallel implementation of the backpropagation algorithm on the IBM GF11 SIMD machine, demonstrating high-performance neural network simulation capabilities with over 900 million connections per second.

Contribution

The paper introduces an efficient parallel implementation of backpropagation on the GF11, including techniques for performance optimization and a detailed analysis of the machine's suitability for neural network simulation.

Findings

Achieved 900 million connections/sec on 356 processors

Projected over 1 billion connections/sec with full processor utilization

Validated GF11's effectiveness for large-scale neural network training

Abstract

Current connectionist simulations require huge computational resources. We describe a neural network simulator for the IBM GF11, an experimental SIMD machine with 566 processors and a peak arithmetic performance of 11 Gigaflops. We present our parallel implementation of the backpropagation learning algorithm, techniques for increasing efficiency, performance measurements on the NetTalk text-to-speech benchmark, and a performance model for the simulator. Our simulator currently runs the back-propagation learning algorithm at 900 million connections per second, where each "connection per second" includes both a forward and backward pass. This figure was obtained on the machine when only 356 processors were working; with all 566 processors operational, our simulation will run at over one billion connections per second. We conclude that the GF11 is well-suited to neural network simulation, and we analyze our use of the machine to determine which features are the most important for high performance.