Analysis of Multilayer Perceptron with Rectifier Linear Unit Activation Function

TL;DR

This paper introduces a multilayer perceptron with a linear activation function, analyzing its temperature and noise robustness, addressing a gap in analog neural network implementations.

Contribution

It presents the design and analysis of a linear activation function-based perceptron, which is less explored compared to sigmoid and tangent functions.

Findings

Perceptron exhibits good performance under various conditions.

Strong durability to temperature changes was observed.

The implementation addresses a key open problem in analog neural networks.

Abstract

The implementation of analog neural network and online analog learning circuits based on memristive crossbar has been intensively explored in recent years. The implementation of various activation functions is important, especially for deep leaning neural networks. There are several implementations of sigmoid and tangent activation function, while the implementation of the neural networks with linear activation functions is an open problem. Therefore, this paper introduces a multilayer perceptron design with linear activation function. The temperature and noise analysis was performed. The perceptron showed a good performance and strong durability to temperature changes.