Cross-Layer Design of Vector-Symbolic Computing: Bridging Cognition and Brain-Inspired Hardware Acceleration

TL;DR

This paper presents a comprehensive cross-layer co-design methodology for vector-symbolic architectures, integrating theoretical principles with hardware implementations to advance brain-inspired computing systems.

Contribution

It introduces a unified framework for hardware-software co-design of VSAs, including a novel in-memory cognition hardware system demonstrating efficiency and scalability.

Findings

Analyzed performance tradeoffs of analog, mixed-signal, and digital VSA hardware implementations.

Proposed a cross-layer design methodology for VSA hardware and software integration.

Developed the first in-memory computing hierarchical cognition hardware system.

Abstract

Vector Symbolic Architectures (VSAs) have been widely deployed in various cognitive applications due to their simple and efficient operations. The widespread adoption of VSAs has, in turn, spurred the development of numerous hardware solutions aimed at optimizing their performance. Despite these advancements, a comprehensive and unified discourse on the convergence of hardware and algorithms in the context of VSAs remains somewhat limited. The paper aims to bridge the gap between theoretical software-level explorations and the development of efficient hardware architectures and emerging technology fabrics for VSAs, providing insights from the co-design aspect for researchers from either side. First, we introduce the principles of vector-symbolic computing, including its core mathematical operations and learning paradigms. Second, we provide an in-depth discussion on hardware technologies for VSAs, analyzing analog, mixed-signal, and digital circuit design styles. We compare hardware implementations of VSAs by carrying out detailed analysis of their performance characteristics and tradeoffs, allowing us to extract design guidelines for the development of arbitrary VSA formulations. Third, we discuss a methodology for cross-layer design of VSAs that identifies synergies across layers and explores key ingredients for hardware/software co-design of VSAs. Finally, as a concrete demonstration of this methodology, we propose the first in-memory computing hierarchical cognition hardware system, showcasing the efficiency, flexibility, and scalability of this co-design approach. The paper concludes with a discussion of open research challenges for future explorations.