Properties of Sparse Distributed Representations and their Application to Hierarchical Temporal Memory

TL;DR

This paper explores the properties of Sparse Distributed Representations (SDRs) in the neocortex and HTM, providing theoretical insights and practical guidelines to enhance understanding of cortical information processing.

Contribution

It introduces core properties of SDRs that support scaling, robustness, and generalization, advancing a unified framework for cortical function modeling.

Findings

SDRs are fundamental to neocortical information representation

Theoretical properties support scalability and robustness of SDRs

Practical guidelines for implementing SDRs in HTM systems

Abstract

Empirical evidence demonstrates that every region of the neocortex represents information using sparse activity patterns. This paper examines Sparse Distributed Representations (SDRs), the primary information representation strategy in Hierarchical Temporal Memory (HTM) systems and the neocortex. We derive a number of properties that are core to scaling, robustness, and generalization. We use the theory to provide practical guidelines and illustrate the power of SDRs as the basis of HTM. Our goal is to help create a unified mathematical and practical framework for SDRs as it relates to cortical function.