Denoising Pre-Training and Data Augmentation Strategies for Enhanced RDF Verbalization with Transformers

TL;DR

This paper improves RDF triple verbalization by combining denoising pre-training and data augmentation strategies with Transformers, significantly enhancing text generation quality for both seen and unseen data categories.

Contribution

It introduces a novel approach that leverages augmented data and denoising pre-training to improve RDF-to-text verbalization with Transformer models.

Findings

BLEU score increases by up to 126.05% for unseen entities

Significant improvement in verbalization quality across categories

Demonstrates effectiveness of data augmentation in RDF verbalization

Abstract

The task of verbalization of RDF triples has known a growth in popularity due to the rising ubiquity of Knowledge Bases (KBs). The formalism of RDF triples is a simple and efficient way to store facts at a large scale. However, its abstract representation makes it difficult for humans to interpret. For this purpose, the WebNLG challenge aims at promoting automated RDF-to-text generation. We propose to leverage pre-trainings from augmented data with the Transformer model using a data augmentation strategy. Our experiment results show a minimum relative increases of 3.73%, 126.05% and 88.16% in BLEU score for seen categories, unseen entities and unseen categories respectively over the standard training.