A Non-commutative Bilinear Model for Answering Path Queries in Knowledge Graphs

TL;DR

This paper introduces BlockHolE, a bilinear knowledge graph embedding model using non-commutative block circulant matrices, enabling natural modeling of composite relations with efficient computation.

Contribution

It proposes a novel non-commutative bilinear model for knowledge graph embeddings that captures composite relations more naturally than existing diagonal models.

Findings

Model allows non-commutative relation matrices.

Efficient Fourier transform-based computation technique.

Flexible parameterization from diagonal to full matrices.

Abstract

Bilinear diagonal models for knowledge graph embedding (KGE), such as DistMult and ComplEx, balance expressiveness and computational efficiency by representing relations as diagonal matrices. Although they perform well in predicting atomic relations, composite relations (relation paths) cannot be modeled naturally by the product of relation matrices, as the product of diagonal matrices is commutative and hence invariant with the order of relations. In this paper, we propose a new bilinear KGE model, called BlockHolE, based on block circulant matrices. In BlockHolE, relation matrices can be non-commutative, allowing composite relations to be modeled by matrix product. The model is parameterized in a way that covers a spectrum ranging from diagonal to full relation matrices. A fast computation technique is developed on the basis of the duality of the Fourier transform of circulant matrices.