Fuse and Mix: MACAM-Enabled Analog Activation for Energy-Efficient Neural Acceleration

TL;DR

This paper introduces MACAM, a magnetic tunnel junction-based analog content-addressable memory, enabling energy-efficient neural acceleration by combining nonlinear activation and A/D conversion, optimized via a differential framework.

Contribution

The work presents a novel MACAM-based analog activation unit that fuses activation and A/D conversion, along with an optimization framework for workload assignment to improve energy efficiency.

Findings

Achieves over 60% energy savings compared to standard activation methods.

Maintains competitive accuracy with reduced A/D conversion energy.

Demonstrates effectiveness on a silicon photonic accelerator.

Abstract

Analog computing has been recognized as a promising low-power alternative to digital counterparts for neural network acceleration. However, conventional analog computing is mainly in a mixed-signal manner. Tedious analog/digital (A/D) conversion cost significantly limits the overall system's energy efficiency. In this work, we devise an efficient analog activation unit with magnetic tunnel junction (MTJ)-based analog content-addressable memory (MACAM), simultaneously realizing nonlinear activation and A/D conversion in a fused fashion. To compensate for the nascent and therefore currently limited representation capability of MACAM, we propose to mix our analog activation unit with digital activation dataflow. A fully differential framework, SuperMixer, is developed to search for an optimized activation workload assignment, adaptive to various activation energy constraints. The effectiveness of our proposed methods is evaluated on a silicon photonic accelerator. Compared to standard activation implementation, our mixed activation system with the searched assignment can achieve competitive accuracy with $>$60% energy saving on A/D conversion and activation.