Electrical Tunable Spintronic Neuron with Trainable Activation Function

TL;DR

This paper introduces a spintronic neuron with a dynamically adjustable activation function, enabling improved neural network training and accuracy without extra energy costs, by exploiting magnetic properties for hardware-based learning.

Contribution

It presents a novel spintronic neuron with tunable activation functions controlled by magnetic properties, allowing hardware-based adaptation during training.

Findings

Improved digit recognition accuracy from 88% to 91.3%.

Activation function slope can be electrically tuned during training.

No additional energy consumption is required for the tunability.

Abstract

Spintronic devices have been widely studied for the hardware realization of artificial neurons. The stochastic switching of magnetic tunnel junction driven by the spin torque is commonly used to produce the sigmoid activation function. However, the shape of the activation function in previous studies is fixed during the training of neural network. This restricts the updating of weights and results in a limited performance. In this work, we exploit the physics behind the spin torque induced magnetization switching to enable the dynamic change of the activation function during the training process. Specifically, the pulse width and magnetic anisotropy can be electrically controlled to change the slope of activation function, which enables a faster or slower change of output required by the backpropagation algorithm. This is also similar to the idea of batch normalization that is widely used in the machine learning. Thus, this work demonstrates that the algorithms are no longer limited to the software implementation. They can in fact be realized by the spintronic hardware using a single device. Finally, we show that the accuracy of hand-written digit recognition can be improved from 88% to 91.3% by using these trainable spintronic neurons without introducing additional energy consumption. Our proposals can stimulate the hardware realization of spintronic neural networks.