A Novel Method for Scalable VLSI Implementation of Hyperbolic Tangent Function

TL;DR

This paper introduces a scalable hardware implementation method for the hyperbolic tangent function, crucial for neural network accelerators, using trigonometric expansion to balance accuracy, performance, and area.

Contribution

It proposes a novel, tunable hardware design based on trigonometric expansion properties for efficient hyperbolic tangent computation in neural network accelerators.

Findings

Achieves low complexity and high accuracy in hardware implementation

Supports scalable precision for different neural network layers

Reduces power and area compared to traditional methods

Abstract

Hyperbolic tangent and Sigmoid functions are used as non-linear activation units in the artificial and deep neural networks. Since, these networks are computationally expensive, customized accelerators are designed for achieving the required performance at lower cost and power. The activation function and MAC units are the key building blocks of these neural networks. A low complexity and accurate hardware implementation of the activation function is required to meet the performance and area targets of such neural network accelerators. Moreover, a scalable implementation is required as the recent studies show that the DNNs may use different precision in different layers. This paper presents a novel method based on trigonometric expansion properties of the hyperbolic function for hardware implementation which can be easily tuned for different accuracy and precision requirements.