Modeling Graph Structure via Relative Position for Text Generation from Knowledge Graphs

TL;DR

Graformer is a Transformer-based model that encodes entire graphs using novel self-attention, capturing global patterns and relation strengths, leading to improved graph-to-text generation with fewer parameters.

Contribution

Introduces Graformer with a new graph self-attention mechanism that considers all node pairs and learns relation importance, enhancing graph encoding for text generation.

Findings

Achieves strong results on AGENDA and WebNLG benchmarks.

Uses fewer parameters than comparable models.

Effectively captures global graph patterns.

Abstract

We present Graformer, a novel Transformer-based encoder-decoder architecture for graph-to-text generation. With our novel graph self-attention, the encoding of a node relies on all nodes in the input graph - not only direct neighbors - facilitating the detection of global patterns. We represent the relation between two nodes as the length of the shortest path between them. Graformer learns to weight these node-node relations differently for different attention heads, thus virtually learning differently connected views of the input graph. We evaluate Graformer on two popular graph-to-text generation benchmarks, AGENDA and WebNLG, where it achieves strong performance while using many fewer parameters than other approaches.