Few-shot Learning for Slot Tagging with Attentive Relational Network

TL;DR

This paper introduces Attentive Relational Network, a novel metric-based learning architecture for slot tagging that leverages pretrained embeddings and attention mechanisms, outperforming existing methods on SNIPS data.

Contribution

The paper proposes a new metric-based learning model, Attentive Relational Network, tailored for slot tagging in NLP, integrating attention and pretrained embeddings.

Findings

Outperforms state-of-the-art metric-based methods on SNIPS dataset

Effectively leverages pretrained embeddings like BERT and ELMO

Demonstrates the effectiveness of attention mechanisms in slot tagging

Abstract

Metric-based learning is a well-known family of methods for few-shot learning, especially in computer vision. Recently, they have been used in many natural language processing applications but not for slot tagging. In this paper, we explore metric-based learning methods in the slot tagging task and propose a novel metric-based learning architecture - Attentive Relational Network. Our proposed method extends relation networks, making them more suitable for natural language processing applications in general, by leveraging pretrained contextual embeddings such as ELMO and BERT and by using attention mechanism. The results on SNIPS data show that our proposed method outperforms other state-of-the-art metric-based learning methods.