Transformer-based Entity Typing in Knowledge Graphs

TL;DR

This paper introduces a Transformer-based approach for entity typing in knowledge graphs, leveraging local, global, and context transformers to improve the inference of entity types by encoding neighbor information.

Contribution

The paper presents a novel Transformer architecture for entity typing that effectively encodes neighbor content and incorporates class membership information, outperforming existing methods.

Findings

TET outperforms state-of-the-art models on real-world datasets.

The combination of local, global, and context transformers enhances entity type inference.

Semantic strengthening of representations improves accuracy.

Abstract

We investigate the knowledge graph entity typing task which aims at inferring plausible entity types. In this paper, we propose a novel Transformer-based Entity Typing (TET) approach, effectively encoding the content of neighbors of an entity. More precisely, TET is composed of three different mechanisms: a local transformer allowing to infer missing types of an entity by independently encoding the information provided by each of its neighbors; a global transformer aggregating the information of all neighbors of an entity into a single long sequence to reason about more complex entity types; and a context transformer integrating neighbors content based on their contribution to the type inference through information exchange between neighbor pairs. Furthermore, TET uses information about class membership of types to semantically strengthen the representation of an entity. Experiments on two real-world datasets demonstrate the superior performance of TET compared to the state-of-the-art.