Levi Graph AMR Parser using Heterogeneous Attention

TL;DR

This paper introduces a transformer-based AMR parser that uses heterogeneous attention over combined input features to predict graph elements, achieving comparable or better accuracy with fewer parameters.

Contribution

It proposes a novel transformer approach that integrates multiple data types for AMR parsing, eliminating the need for biaffine decoders and reducing model complexity.

Findings

Achieves state-of-the-art or comparable accuracy on AMR 2.0 and 3.0 datasets.

Uses significantly fewer parameters than previous models.

Demonstrates effectiveness of heterogeneous attention in graph prediction.

Abstract

Coupled with biaffine decoders, transformers have been effectively adapted to text-to-graph transduction and achieved state-of-the-art performance on AMR parsing. Many prior works, however, rely on the biaffine decoder for either or both arc and label predictions although most features used by the decoder may be learned by the transformer already. This paper presents a novel approach to AMR parsing by combining heterogeneous data (tokens, concepts, labels) as one input to a transformer to learn attention, and use only attention matrices from the transformer to predict all elements in AMR graphs (concepts, arcs, labels). Although our models use significantly fewer parameters than the previous state-of-the-art graph parser, they show similar or better accuracy on AMR 2.0 and 3.0.