A Mathematical Formalization of Hierarchical Temporal Memory's Spatial Pooler

TL;DR

This paper develops a comprehensive mathematical framework for the spatial pooler component of hierarchical temporal memory, enabling better understanding and application in prediction, classification, and feature learning tasks.

Contribution

It introduces a unifying mathematical formalization of HTM's spatial pooler, including a maximum likelihood estimator and analysis of boosting mechanisms, linking it to known algorithms.

Findings

SP can be used for feature learning with proper parameters

Boosting is mainly relevant in early iterations

SP is applicable for classification and dimensionality reduction

Abstract

Hierarchical temporal memory (HTM) is an emerging machine learning algorithm, with the potential to provide a means to perform predictions on spatiotemporal data. The algorithm, inspired by the neocortex, currently does not have a comprehensive mathematical framework. This work brings together all aspects of the spatial pooler (SP), a critical learning component in HTM, under a single unifying framework. The primary learning mechanism is explored, where a maximum likelihood estimator for determining the degree of permanence update is proposed. The boosting mechanisms are studied and found to be only relevant during the initial few iterations of the network. Observations are made relating HTM to well-known algorithms such as competitive learning and attribute bagging. Methods are provided for using the SP for classification as well as dimensionality reduction. Empirical evidence verifies that given the proper parameterizations, the SP may be used for feature learning.