GPT-too: A language-model-first approach for AMR-to-text generation

TL;DR

This paper introduces GPT-too, a novel AMR-to-text generation method that leverages pre-trained language models and cycle consistency re-scoring, outperforming previous techniques on standard datasets and confirmed by human evaluations.

Contribution

It presents a simple yet effective approach combining pre-trained models with cycle consistency re-scoring for improved AMR-to-text generation.

Findings

Outperforms all previous methods on LDC2017T10 dataset

Achieves superior results in standard and human evaluations

Demonstrates the effectiveness of language-model-first strategies

Abstract

Meaning Representations (AMRs) are broad-coverage sentence-level semantic graphs. Existing approaches to generating text from AMR have focused on training sequence-to-sequence or graph-to-sequence models on AMR annotated data only. In this paper, we propose an alternative approach that combines a strong pre-trained language model with cycle consistency-based re-scoring. Despite the simplicity of the approach, our experimental results show these models outperform all previous techniques on the English LDC2017T10dataset, including the recent use of transformer architectures. In addition to the standard evaluation metrics, we provide human evaluation experiments that further substantiate the strength of our approach.