Noise mitigation strategies in physical feedforward neural networks

TL;DR

This paper introduces analytical and practical noise mitigation strategies for physical neural networks, including intra-layer connection design, ghost neurons, and pooling, significantly improving signal quality and classification accuracy in hardware implementations.

Contribution

It provides a comprehensive analytical framework and novel strategies for noise suppression in physical neural networks, enhancing hardware efficiency and performance.

Findings

Achieved a 4-fold increase in output SNR in MNIST classification.

Demonstrated near noise-free accuracy with combined noise mitigation strategies.

Analytically proved noise suppression via specific intra-layer connection properties.

Abstract

Physical neural networks are promising candidates for next generation artificial intelligence hardware. In such architectures, neurons and connections are physically realized and do not leverage digital concepts with their practically infinite signal-to-noise ratio to encode, transduce and transform information. They therefore are prone to noise with a variety of statistical and architectural properties, and effective strategies leveraging network-inherent assets to mitigate noise in an hardware-efficient manner are important in the pursuit of next generation neural network hardware. Based on analytical derivations, we here introduce and analyse a variety of different noise-mitigation approaches. We analytically show that intra-layer connections in which the connection matrix's squared mean exceeds the mean of its square fully suppresses uncorrelated noise. We go beyond and develop two synergistic strategies for noise that is uncorrelated and correlated across populations of neurons. First, we introduce the concept of ghost neurons, where each group of neurons perturbed by correlated noise has a negative connection to a single neuron, yet without receiving any input information. Secondly, we show that pooling of neuron populations is an efficient approach to suppress uncorrelated noise. As such, we developed a general noise mitigation strategy leveraging the statistical properties of the different noise terms most relevant in analogue hardware. Finally, we demonstrate the effectiveness of this combined approach for trained neural network classifying the MNIST handwritten digits, for which we achieve a 4-fold improvement of the output signal-to-noise ratio and increase the classification accuracy almost to the level of the noise-free network.