Tree Decomposition Attention for AMR-to-Text Generation

TL;DR

This paper introduces a novel tree decomposition attention mechanism for AMR-to-text generation, improving the encoding of semantic graphs by constraining self-attention with hierarchical tree structures, leading to better translation quality.

Contribution

It proposes a tree decomposition-based attention method that enhances graph encoding in AMR-to-text tasks, outperforming standard self-attention models.

Findings

Outperforms baseline by 1.6 BLEU

Outperforms baseline by 1.8 chrF++

Uses dynamic programming for tree selection

Abstract

Text generation from AMR requires mapping a semantic graph to a string that it annotates. Transformer-based graph encoders, however, poorly capture vertex dependencies that may benefit sequence prediction. To impose order on an encoder, we locally constrain vertex self-attention using a graph's tree decomposition. Instead of forming a full query-key bipartite graph, we restrict attention to vertices in parent, subtree, and same-depth bags of a vertex. This hierarchical context lends both sparsity and structure to vertex state updates. We apply dynamic programming to derive a forest of tree decompositions, choosing the most structurally similar tree to the AMR. Our system outperforms a self-attentive baseline by 1.6 BLEU and 1.8 chrF++.