Row-less Universal Schema

TL;DR

This paper introduces a 'row-less' universal schema model that generalizes to unseen entity pairs by using relation types and attention mechanisms, improving knowledge base construction and information extraction.

Contribution

It proposes a novel 'row-less' approach that removes explicit entity pair representations, enabling better generalization to unseen data compared to prior models.

Findings

Matches performance of explicit entity pair models on FB15k-237

Performs well on unseen entity pairs during training

Uses attention over relation types for entity pair representation

Abstract

Universal schema jointly embeds knowledge bases and textual patterns to reason about entities and relations for automatic knowledge base construction and information extraction. In the past, entity pairs and relations were represented as learned vectors with compatibility determined by a scoring function, limiting generalization to unseen text patterns and entities. Recently, 'column-less' versions of Universal Schema have used compositional pattern encoders to generalize to all text patterns. In this work we take the next step and propose a 'row-less' model of universal schema, removing explicit entity pair representations. Instead of learning vector representations for each entity pair in our training set, we treat an entity pair as a function of its relation types. In experimental results on the FB15k-237 benchmark we demonstrate that we can match the performance of a comparable model with explicit entity pair representations using a model of attention over relation types. We further demonstrate that the model per- forms with nearly the same accuracy on entity pairs never seen during training.