Neural Semantic Parsing by Character-based Translation: Experiments with Abstract Meaning Representations

TL;DR

This paper demonstrates that a character-based sequence-to-sequence neural model, with several enhancements, can achieve state-of-the-art accuracy in semantic parsing of sentences into Abstract Meaning Representations, using a simple approach.

Contribution

The paper introduces a straightforward character-level translation method for neural semantic parsing and shows how various techniques significantly improve performance to reach state-of-the-art results.

Findings

Baseline accuracy of 53.1 F-score achieved.

Reordering AMR branches improves to 58.3.

Adding part-of-speech tags yields 57.2 F-score.

Abstract

We evaluate the character-level translation method for neural semantic parsing on a large corpus of sentences annotated with Abstract Meaning Representations (AMRs). Using a sequence-to-sequence model, and some trivial preprocessing and postprocessing of AMRs, we obtain a baseline accuracy of 53.1 (F-score on AMR-triples). We examine five different approaches to improve this baseline result: (i) reordering AMR branches to match the word order of the input sentence increases performance to 58.3; (ii) adding part-of-speech tags (automatically produced) to the input shows improvement as well (57.2); (iii) So does the introduction of super characters (conflating frequent sequences of characters to a single character), reaching 57.4; (iv) optimizing the training process by using pre-training and averaging a set of models increases performance to 58.7; (v) adding silver-standard training data obtained by an off-the-shelf parser yields the biggest improvement, resulting in an F-score of 64.0. Combining all five techniques leads to an F-score of 71.0 on holdout data, which is state-of-the-art in AMR parsing. This is remarkable because of the relative simplicity of the approach.