DocAMR: Multi-Sentence AMR Representation and Evaluation

TL;DR

This paper introduces a unified multi-sentence AMR graph representation and an improved evaluation method, enabling better parsing and comparison of document-level semantic graphs.

Contribution

It proposes a simple algorithm for deriving unified document-level AMR graphs and enhances the Smatch metric for large graphs, establishing a strong baseline for future research.

Findings

New unified graph representation for multi-sentence AMR

Enhanced Smatch metric for document-level graphs

Baseline pipeline combining top AMR parser and coreference resolution

Abstract

Despite extensive research on parsing of English sentences into Abstraction Meaning Representation (AMR) graphs, which are compared to gold graphs via the Smatch metric, full-document parsing into a unified graph representation lacks well-defined representation and evaluation. Taking advantage of a super-sentential level of coreference annotation from previous work, we introduce a simple algorithm for deriving a unified graph representation, avoiding the pitfalls of information loss from over-merging and lack of coherence from under-merging. Next, we describe improvements to the Smatch metric to make it tractable for comparing document-level graphs, and use it to re-evaluate the best published document-level AMR parser. We also present a pipeline approach combining the top performing AMR parser and coreference resolution systems, providing a strong baseline for future research.