Neuromorphic Architecture for the Hierarchical Temporal Memory

TL;DR

This paper presents a comprehensive hardware architecture for Hierarchical Temporal Memory (HTM), demonstrating high accuracy and significant speedup on datasets like MNIST and EUNF, paving the way for hardware-based self-learning systems.

Contribution

The paper introduces a full-scale HTM hardware architecture with synthetic synapses, achieving high accuracy and speed, and validating its effectiveness on multiple datasets with noise.

Findings

91.16% accuracy on MNIST

90% accuracy on noisy EUNF

1364X speedup over software implementation

Abstract

A biomimetic machine intelligence algorithm, that holds promise in creating invariant representations of spatiotemporal input streams is the hierarchical temporal memory (HTM). This unsupervised online algorithm has been demonstrated on several machine-learning tasks, including anomaly detection. Significant effort has been made in formalizing and applying the HTM algorithm to different classes of problems. There are few early explorations of the HTM hardware architecture, especially for the earlier version of the spatial pooler of HTM algorithm. In this article, we present a full-scale HTM architecture for both spatial pooler and temporal memory. Synthetic synapse design is proposed to address the potential and dynamic interconnections occurring during learning. The architecture is interweaved with parallel cells and columns that enable high processing speed for the HTM. The proposed architecture is verified for two different datasets: MNIST and the European number plate font (EUNF), with and without the presence of noise. The spatial pooler architecture is synthesized on Xilinx ZYNQ-7, with 91.16% classification accuracy for MNIST and 90\% accuracy for EUNF, with noise. For the temporal memory sequence prediction, first and second order predictions are observed for a 5-number long sequence generated from EUNF dataset and 95% accuracy is obtained. Moreover, the proposed hardware architecture offers 1364X speedup over the software realization. These results indicate that the proposed architecture can serve as a digital core to build the HTM in hardware and eventually as a standalone self-learning system.