Augmenting Compositional Models for Knowledge Base Completion Using Gradient Representations

TL;DR

This paper introduces Gradient Graphs, a novel approach that optimizes triplet embeddings in knowledge bases using harmonic grammar-inspired techniques, improving performance and interpretability of compositional models.

Contribution

It presents Gradient Graphs, a new method that enhances knowledge base completion by optimizing triplet embeddings with well-formedness constraints, adding interpretability and performance gains.

Findings

Significant improvements in knowledge base completion accuracy.

Triplet embeddings exhibit interpretable properties.

Enhanced inference capabilities with Gradient Graphs.

Abstract

Neural models of Knowledge Base data have typically employed compositional representations of graph objects: entity and relation embeddings are systematically combined to evaluate the truth of a candidate Knowedge Base entry. Using a model inspired by Harmonic Grammar, we propose to tokenize triplet embeddings by subjecting them to a process of optimization with respect to learned well-formedness conditions on Knowledge Base triplets. The resulting model, known as Gradient Graphs, leads to sizable improvements when implemented as a companion to compositional models. Also, we show that the "supracompositional" triplet token embeddings it produces have interpretable properties that prove helpful in performing inference on the resulting triplet representations.