A Factorization Machine Framework for Testing Bigram Embeddings in Knowledgebase Completion

TL;DR

This paper introduces a factorization machine framework to test bigram embeddings for knowledge base completion, showing that pairwise embeddings can improve link prediction accuracy over traditional compositional models.

Contribution

It presents a novel application of factorization machines to bigram embeddings in knowledge base completion, demonstrating their effectiveness over standard compositional approaches.

Findings

Bigram embeddings improve link prediction accuracy.

Factorization machine models outperform compositional models.

Relevance of various bigram types varies across datasets.

Abstract

Embedding-based Knowledge Base Completion models have so far mostly combined distributed representations of individual entities or relations to compute truth scores of missing links. Facts can however also be represented using pairwise embeddings, i.e. embeddings for pairs of entities and relations. In this paper we explore such bigram embeddings with a flexible Factorization Machine model and several ablations from it. We investigate the relevance of various bigram types on the fb15k237 dataset and find relative improvements compared to a compositional model.