A Time-domain Analog Weighted-sum Calculation Model for Extremely Low Power VLSI Implementation of Multi-layer Neural Networks

TL;DR

This paper introduces a novel time-domain analog weighted-sum calculation model for ultra-low power multi-layer neural networks, implemented with VLSI circuits that operate without operational amplifiers, achieving high energy efficiency.

Contribution

The paper proposes a new time-domain analog model for neural network weighted sums and demonstrates a low-power VLSI implementation with simulation results showing superior energy efficiency.

Findings

Achieved 290 TOPS/W energy efficiency in simulations.

Proposed circuits operate without operational amplifiers, reducing power.

Potential to exceed 1,000 TOPS/W with advanced technology.

Abstract

A time-domain analog weighted-sum calculation model is proposed based on an integrate-and-fire-type spiking neuron model. The proposed calculation model is applied to multi-layer feedforward networks, in which weighted summations with positive and negative weights are separately performed in each layer and summation results are then fed into the next layers without their subtraction operation. We also propose very large-scale integrated (VLSI) circuits to implement the proposed model. Unlike the conventional analog voltage or current mode circuits, the time-domain analog circuits use transient operation in charging/discharging processes to capacitors. Since the circuits can be designed without operational amplifiers, they can operate with extremely low power consumption. However, they have to use very high resistance devices on the order of G$\rm \Omega$. We designed a proof-of-concept (PoC) CMOS VLSI chip to verify weighted-sum operation with the same weights and evaluated it by post-layout circuit simulation using 250-nm fabrication technology. High resistance operation was realized by using the subthreshold operation region of MOS transistors. Simulation results showed that energy efficiency for the weighted-sum calculation was 290~TOPS/W, more than one order of magnitude higher than that in state-of-the-art digital AI processors, even though the minimum width of interconnection used in the PoC chip was several times larger than that in such digital processors. If state-of-the-art VLSI technology is used to implement the proposed model, an energy efficiency of more than 1,000~TOPS/W will be possible. For practical applications, development of emerging analog memory devices such as ferroelectric-gate FETs is necessary.