LARS-VSA: A Vector Symbolic Architecture For Learning with Abstract Rules

TL;DR

LARS-VSA introduces a neuro-symbolic architecture using hyperdimensional computing and a relational bottleneck to improve symbolic reasoning efficiency and accuracy with limited data, overcoming interference issues in object representations.

Contribution

It adapts the relational bottleneck to high-dimensional space with explicit vector binding and a novel attention mechanism, enhancing efficiency and robustness in symbolic learning.

Findings

Outperforms state-of-the-art in efficiency and accuracy

Robust to interference in object representations

Effective with limited training samples

Abstract

Human cognition excels at symbolic reasoning, deducing abstract rules from limited samples. This has been explained using symbolic and connectionist approaches, inspiring the development of a neuro-symbolic architecture that combines both paradigms. In parallel, recent studies have proposed the use of a "relational bottleneck" that separates object-level features from abstract rules, allowing learning from limited amounts of data . While powerful, it is vulnerable to the curse of compositionality meaning that object representations with similar features tend to interfere with each other. In this paper, we leverage hyperdimensional computing, which is inherently robust to such interference to build a compositional architecture. We adapt the "relational bottleneck" strategy to a high-dimensional space, incorporating explicit vector binding operations between symbols and relational representations. Additionally, we design a novel high-dimensional attention mechanism that leverages this relational representation. Our system benefits from the low overhead of operations in hyperdimensional space, making it significantly more efficient than the state of the art when evaluated on a variety of test datasets, while maintaining higher or equal accuracy.