Ask2Transformers: Zero-Shot Domain labelling with Pre-trained Language Models

TL;DR

This paper introduces Ask2Transformers, a zero-shot system that leverages pre-trained language models to assign domain labels to WordNet synsets without supervision, achieving state-of-the-art results.

Contribution

It presents a novel zero-shot approach using pre-trained language models for domain labeling of WordNet synsets, without relying on task-specific training.

Findings

Achieves new state-of-the-art performance on English WordNet dataset

Demonstrates effectiveness of pre-trained models for zero-shot domain labeling

Flexible system not restricted to specific domain labels

Abstract

In this paper we present a system that exploits different pre-trained Language Models for assigning domain labels to WordNet synsets without any kind of supervision. Furthermore, the system is not restricted to use a particular set of domain labels. We exploit the knowledge encoded within different off-the-shelf pre-trained Language Models and task formulations to infer the domain label of a particular WordNet definition. The proposed zero-shot system achieves a new state-of-the-art on the English dataset used in the evaluation.