Mind the Labels: Describing Relations in Knowledge Graphs With Pretrained Models

TL;DR

This paper investigates how pretrained language models verbalize relations in knowledge graphs, highlighting the importance of diverse, clear labels for better generalization to unseen relations in data-to-text generation.

Contribution

It introduces a new dataset of 1,522 relations and demonstrates that training with diverse relation labels improves models' ability to verbalize unseen relations.

Findings

Models struggle with ambiguous relation labels.

Diverse relation labels enhance generalization to new relations.

Training with clear, varied labels improves robustness.

Abstract

Pretrained language models (PLMs) for data-to-text (D2T) generation can use human-readable data labels such as column headings, keys, or relation names to generalize to out-of-domain examples. However, the models are well-known in producing semantically inaccurate outputs if these labels are ambiguous or incomplete, which is often the case in D2T datasets. In this paper, we expose this issue on the task of descibing a relation between two entities. For our experiments, we collect a novel dataset for verbalizing a diverse set of 1,522 unique relations from three large-scale knowledge graphs (Wikidata, DBPedia, YAGO). We find that although PLMs for D2T generation expectedly fail on unclear cases, models trained with a large variety of relation labels are surprisingly robust in verbalizing novel, unseen relations. We argue that using data with a diverse set of clear and meaningful labels is key to training D2T generation systems capable of generalizing to novel domains.