Syntax-Aware Graph-to-Graph Transformer for Semantic Role Labelling

TL;DR

This paper introduces a syntax-aware Transformer model for semantic role labeling that integrates syntactic structures directly into the self-attention mechanism, improving performance on multiple datasets.

Contribution

The novel SynG2G-Tr model encodes syntactic relations as embeddings within Transformer self-attention, enhancing SRL performance with a flexible, structure-informed approach.

Findings

Outperforms previous methods on CoNLL 2005 and 2009 datasets.

Effective in both in-domain and out-of-domain settings.

Utilizes syntactic structure to guide attention patterns.

Abstract

Recent models have shown that incorporating syntactic knowledge into the semantic role labelling (SRL) task leads to a significant improvement. In this paper, we propose Syntax-aware Graph-to-Graph Transformer (SynG2G-Tr) model, which encodes the syntactic structure using a novel way to input graph relations as embeddings, directly into the self-attention mechanism of Transformer. This approach adds a soft bias towards attention patterns that follow the syntactic structure but also allows the model to use this information to learn alternative patterns. We evaluate our model on both span-based and dependency-based SRL datasets, and outperform previous alternative methods in both in-domain and out-of-domain settings, on CoNLL 2005 and CoNLL 2009 datasets.