Cortical Processing with Thermodynamic-RAM

TL;DR

This paper introduces Thermodynamic-RAM, a memristor-based neural processing unit inspired by AHaH computing, demonstrating its design, implementation, and improved machine learning performance.

Contribution

It presents the design, formal instruction set, emulator, and application of Thermodynamic-RAM, advancing memristor-based neural hardware for machine learning.

Findings

Successful porting of multiple machine learning benchmarks to kT-RAM

Achieved a 99.5% F1 score on MNIST with synaptic healing

Demonstrated advantages of kT-RAM over traditional architectures

Abstract

AHaH computing forms a theoretical framework from which a biologically-inspired type of computing architecture can be built where, unlike von Neumann systems, memory and processor are physically combined. In this paper we report on an incremental step beyond the theoretical framework of AHaH computing toward the development of a memristor-based physical neural processing unit (NPU), which we call Thermodynamic-RAM (kT-RAM). While the power consumption and speed dominance of such an NPU over von Neumann architectures for machine learning applications is well appreciated, Thermodynamic-RAM offers several advantages over other hardware approaches to adaptation and learning. Benefits include general-purpose use, a simple yet flexible instruction set and easy integration into existing digital platforms. We present a high level design of kT-RAM and a formal definition of its instruction set. We report the completion of a kT-RAM emulator and the successful port of all previous machine learning benchmark applications including unsupervised clustering, supervised and unsupervised classification, complex signal prediction, unsupervised robotic actuation and combinatorial optimization. Lastly, we extend a previous MNIST hand written digits benchmark application, to show that an extra step of reading the synaptic states of AHaH nodes during the train phase (healing) alone results in plasticity that improves the classifier's performance, bumping our best F1 score up to 99.5%.