Representing Objects, Relations, and Sequences
Stephen I. Gallant, T. Wendy Okaywe

TL;DR
This paper introduces a new Vector Symbolic Architecture called MBAT that uses matrix multiplication for binding, satisfying constraints for representing complex structures, and discusses its implications for machine learning and neural systems.
Contribution
The paper develops a novel VSA, MBAT, employing matrix binding with random elements, and demonstrates its theoretical and practical advantages for representing complex structures in machine learning.
Findings
MBAT satisfies all theoretical constraints for VSAs.
Matrix binding enables efficient representation of complex structures.
Simulations suggest MBAT is suitable for real-world applications.
Abstract
Vector Symbolic Architectures (VSAs) are high-dimensional vector representations of objects (eg., words, image parts), relations (eg., sentence structures), and sequences for use with machine learning algorithms. They consist of a vector addition operator for representing a collection of unordered objects, a Binding operator for associating groups of objects, and a methodology for encoding complex structures. We first develop Constraints that machine learning imposes upon VSAs: for example, similar structures must be represented by similar vectors. The constraints suggest that current VSAs should represent phrases ("The smart Brazilian girl") by binding sums of terms, in addition to simply binding the terms directly. We show that matrix multiplication can be used as the binding operator for a VSA, and that matrix elements can be chosen at random. A consequence for living systems is…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNeural Networks and Applications · Machine Learning in Bioinformatics · Topic Modeling
