A logical re-conception of neural networks: Hamiltonian bitwise part-whole architecture

TL;DR

This paper proposes a novel neural network architecture based on a Hamiltonian graph operator that intrinsically encodes relations like part-whole, enabling symbolic-like reasoning with low-precision arithmetic and scalable computation.

Contribution

It introduces a Hamiltonian-based graph architecture that directly encodes relations and produces hierarchical, symbolic-like representations within neural networks.

Findings

Successfully encodes relational structures such as part-of and next-to.

Produces hierarchical representations enabling abductive inference.

Operates with low-precision arithmetic and scales linearly with data size.

Abstract

We introduce a simple initial working system in which relations (such as part-whole) are directly represented via an architecture with operating and learning rules fundamentally distinct from standard artificial neural network methods. Arbitrary data are straightforwardly encoded as graphs whose edges correspond to codes from a small fixed primitive set of elemental pairwise relations, such that simple relational encoding is not an add-on, but occurs intrinsically within the most basic components of the system. A novel graph-Hamiltonian operator calculates energies among these encodings, with ground states denoting simultaneous satisfaction of all relation constraints among graph vertices. The method solely uses radically low-precision arithmetic; computational cost is correspondingly low, and scales linearly with the number of edges in the data. The resulting unconventional architecture can process standard ANN examples, but also produces representations that exhibit characteristics of symbolic computation. Specifically, the method identifies simple logical relational structures in these data (part-of; next-to), building hierarchical representations that enable abductive inferential steps generating relational position-based encodings, rather than solely statistical representations. Notably, an equivalent set of ANN operations are derived, identifying a special case of embedded vector encodings that may constitute a useful approach to current work in higher-level semantic representation. The very simple current state of the implemented system invites additional tools and improvements.