Prompt-Learning for Fine-Grained Entity Typing

TL;DR

This paper explores prompt-learning techniques for fine-grained entity typing across various data regimes, demonstrating significant improvements over traditional fine-tuning, especially in low-data scenarios.

Contribution

It introduces a prompt-learning pipeline with entity-specific verbalizers and templates, and proposes a self-supervised strategy for zero-shot entity typing.

Findings

Prompt-learning outperforms fine-tuning in low-data settings.

The proposed methods achieve state-of-the-art results on three benchmarks.

Zero-shot approach effectively summarizes entity type information.

Abstract

As an effective approach to tune pre-trained language models (PLMs) for specific tasks, prompt-learning has recently attracted much attention from researchers. By using \textit{cloze}-style language prompts to stimulate the versatile knowledge of PLMs, prompt-learning can achieve promising results on a series of NLP tasks, such as natural language inference, sentiment classification, and knowledge probing. In this work, we investigate the application of prompt-learning on fine-grained entity typing in fully supervised, few-shot and zero-shot scenarios. We first develop a simple and effective prompt-learning pipeline by constructing entity-oriented verbalizers and templates and conducting masked language modeling. Further, to tackle the zero-shot regime, we propose a self-supervised strategy that carries out distribution-level optimization in prompt-learning to automatically summarize the information of entity types. Extensive experiments on three fine-grained entity typing benchmarks (with up to 86 classes) under fully supervised, few-shot and zero-shot settings show that prompt-learning methods significantly outperform fine-tuning baselines, especially when the training data is insufficient.