Hey Pentti, We Did (More of) It!: A Vector-Symbolic Lisp With Residue Arithmetic

TL;DR

This paper introduces a novel vector-symbolic Lisp system using Frequency-domain Holographic Reduced Representations and Residue Hyperdimensional Computing, enhancing neural network interpretability and expressivity for structured, Turing-complete representations.

Contribution

It extends VSA encoding of Lisp with arithmetic primitives via RHC, enabling neural networks to handle structured, interpretable, and Turing-complete representations.

Findings

Enhanced neural network expressivity with structured representations

Potential for more general intelligent agents

Increased interpretability of neural network states

Abstract

Using Frequency-domain Holographic Reduced Representations (FHRRs), we extend a Vector-Symbolic Architecture (VSA) encoding of Lisp 1.5 with primitives for arithmetic operations using Residue Hyperdimensional Computing (RHC). Encoding a Turing-complete syntax over a high-dimensional vector space increases the expressivity of neural network states, enabling network states to contain arbitrarily structured representations that are inherently interpretable. We discuss the potential applications of the VSA encoding in machine learning tasks, as well as the importance of encoding structured representations and designing neural networks whose behavior is sensitive to the structure of their representations in virtue of attaining more general intelligent agents than exist at present.