Character-level Representations Improve DRS-based Semantic Parsing Even in the Age of BERT

TL;DR

This paper demonstrates that integrating character-level representations with contextual language models significantly enhances Discourse Representation Structure parsing across languages and datasets, outperforming other linguistic feature additions.

Contribution

It introduces a robust method to incorporate character-level info into seq2seq models, improving semantic parsing performance across multiple languages and datasets.

Findings

Character-level representations boost parsing accuracy.

Improvements are consistent across languages and datasets.

Semantic analysis shows better handling of specific phenomena.

Abstract

We combine character-level and contextual language model representations to improve performance on Discourse Representation Structure parsing. Character representations can easily be added in a sequence-to-sequence model in either one encoder or as a fully separate encoder, with improvements that are robust to different language models, languages and data sets. For English, these improvements are larger than adding individual sources of linguistic information or adding non-contextual embeddings. A new method of analysis based on semantic tags demonstrates that the character-level representations improve performance across a subset of selected semantic phenomena.