Universal Representation Learning of Knowledge Bases by Jointly Embedding Instances and Ontological Concepts

TL;DR

This paper introduces JOIE, a novel two-view knowledge graph embedding model that jointly represents ontological concepts and instances, improving performance on multiple tasks by capturing cross-view and intra-view relationships.

Contribution

JOIE is the first model to simultaneously embed both ontology and instance views of knowledge bases with cross-view and intra-view modeling, enabling better multi-view knowledge representation.

Findings

JOIE significantly outperforms previous models on triple prediction tasks.

JOIE improves ontology population accuracy.

The model effectively extends to entity typing applications.

Abstract

Many large-scale knowledge bases simultaneously represent two views of knowledge graphs (KGs): an ontology view for abstract and commonsense concepts, and an instance view for specific entities that are instantiated from ontological concepts. Existing KG embedding models, however, merely focus on representing one of the two views alone. In this paper, we propose a novel two-view KG embedding model, JOIE, with the goal to produce better knowledge embedding and enable new applications that rely on multi-view knowledge. JOIE employs both cross-view and intra-view modeling that learn on multiple facets of the knowledge base. The cross-view association model is learned to bridge the embeddings of ontological concepts and their corresponding instance-view entities. The intra-view models are trained to capture the structured knowledge of instance and ontology views in separate embedding spaces, with a hierarchy-aware encoding technique enabled for ontologies with hierarchies. We explore multiple representation techniques for the two model components and investigate with nine variants of JOIE. Our model is trained on large-scale knowledge bases that consist of massive instances and their corresponding ontological concepts connected via a (small) set of cross-view links. Experimental results on public datasets show that the best variant of JOIE significantly outperforms previous models on instance-view triple prediction task as well as ontology population on ontologyview KG. In addition, our model successfully extends the use of KG embeddings to entity typing with promising performance.