Using Floating Gate Memory to Train Ideal Accuracy Neural Networks

TL;DR

This paper demonstrates that floating gate SONOS transistors can be used to train neural networks with near-ideal accuracy, offering significant improvements in energy efficiency, speed, and area over traditional SRAM-based hardware.

Contribution

It introduces a novel neural training approach using SONOS transistors operating in the subthreshold regime, achieving high accuracy and efficiency improvements.

Findings

Achieves near-ideal accuracy on MNIST with multiple devices per weight.

Single-device implementation reaches within 1% of ideal accuracy.

Energy efficiency increases by 120X compared to SRAM-based ASICs.

Abstract

Floating gate SONOS (Silicon-Oxygen-Nitrogen-Oxygen-Silicon) transistors can be used to train neural networks to ideal accuracies that match those of floating point digital weights on the MNIST dataset when using multiple devices to represent a weight or within 1% of ideal accuracy when using a single device. This is enabled by operating devices in the subthreshold regime, where they exhibit symmetric write nonlinearities. A neural training accelerator core based on SONOS with a single device per weight would increase energy efficiency by 120X, operate 2.1X faster and require 5X lower area than an optimized SRAM based ASIC.