GNN-SL: Sequence Labeling Based on Nearest Examples via GNN

TL;DR

This paper introduces GNN-SL, a novel sequence labeling method that uses graph neural networks to incorporate similar training examples, significantly improving performance on tasks like NER, POS tagging, and Chinese Word Segmentation.

Contribution

The paper proposes a new GNN-based approach for sequence labeling that effectively leverages retrieved similar examples to enhance prediction accuracy, especially for long-tail cases.

Findings

Achieved state-of-the-art results on multiple sequence labeling tasks.

Significantly improved performance on Chinese Word Segmentation.

Demonstrated effectiveness across NER and POS datasets.

Abstract

To better handle long-tail cases in the sequence labeling (SL) task, in this work, we introduce graph neural networks sequence labeling (GNN-SL), which augments the vanilla SL model output with similar tagging examples retrieved from the whole training set. Since not all the retrieved tagging examples benefit the model prediction, we construct a heterogeneous graph, and leverage graph neural networks (GNNs) to transfer information between the retrieved tagging examples and the input word sequence. The augmented node which aggregates information from neighbors is used to do prediction. This strategy enables the model to directly acquire similar tagging examples and improves the general quality of predictions. We conduct a variety of experiments on three typical sequence labeling tasks: Named Entity Recognition (NER), Part of Speech Tagging (POS), and Chinese Word Segmentation (CWS) to show the significant performance of our GNN-SL. Notably, GNN-SL achieves SOTA results of 96.9 (+0.2) on PKU, 98.3 (+0.4) on CITYU, 98.5 (+0.2) on MSR, and 96.9 (+0.2) on AS for the CWS task, and results comparable to SOTA performances on NER datasets, and POS datasets.