Motif Learning in Knowledge Graphs Using Trajectories Of Differential Equations

TL;DR

This paper introduces a novel knowledge graph embedding method using neural differential equations to model complex geometric structures, improving the preservation of graph motifs and reducing inference errors.

Contribution

It proposes a neuro differential KGE that embeds entities via ODE trajectories on manifolds, capturing complex structures better than flat geometry models.

Findings

ODE-based embeddings preserve graph motifs effectively

The method outperforms state-of-the-art models on synthetic and real datasets

Reduces incorrect inferences in knowledge graph link prediction

Abstract

Knowledge Graph Embeddings (KGEs) have shown promising performance on link prediction tasks by mapping the entities and relations from a knowledge graph into a geometric space (usually a vector space). Ultimately, the plausibility of the predicted links is measured by using a scoring function over the learned embeddings (vectors). Therefore, the capability in preserving graph characteristics including structural aspects and semantics highly depends on the design of the KGE, as well as the inherited abilities from the underlying geometry. Many KGEs use the flat geometry which renders them incapable of preserving complex structures and consequently causes wrong inferences by the models. To address this problem, we propose a neuro differential KGE that embeds nodes of a KG on the trajectories of Ordinary Differential Equations (ODEs). To this end, we represent each relation (edge) in a KG as a vector field on a smooth Riemannian manifold. We specifically parameterize ODEs by a neural network to represent various complex shape manifolds and more importantly complex shape vector fields on the manifold. Therefore, the underlying embedding space is capable of getting various geometric forms to encode complexity in subgraph structures with different motifs. Experiments on synthetic and benchmark dataset as well as social network KGs justify the ODE trajectories as a means to structure preservation and consequently avoiding wrong inferences over state-of-the-art KGE models.