Graph Embeddings via Tensor Products and Approximately Orthonormal Codes

TL;DR

This paper introduces a novel dynamic graph representation based on tensor products and hyperdimensional computing principles, analyzing its theoretical properties and capacity to efficiently encode large sparse graphs.

Contribution

It presents a new tensor product-based graph embedding method, characterizes its theoretical properties, and compares its capacity with existing hyperdimensional coding schemes.

Findings

Tensor product is the most general binding operation in HDC.

The method's memory capacity scales similarly to Hadamard-Rademacher schemes.

It acts as a pseudo-orthogonal generalization of adjacency matrices.

Abstract

We propose a dynamic graph representation method, showcasing its rich representational capacity and establishing some of its theoretical properties. Our representation falls under the bind-and-sum approach in hyperdimensional computing (HDC), and we show that the tensor product is the most general binding operation that respects the superposition principle employed in HDC. We also establish some precise results characterizing the behavior of our method, including a memory vs. size analysis of how our representation's size must scale with the number of edges in order to retain accurate graph operations. True to its HDC roots, we also compare our graph representation to another typical HDC representation, the Hadamard-Rademacher scheme, showing that these two graph representations have the same memory-capacity scaling. We establish a link to adjacency matrices, showing that our method is a pseudo-orthogonal generalization of adjacency matrices. In light of this, we briefly discuss its applications toward a dynamic compressed representation of large sparse graphs.