Scalar Adjective Identification and Multilingual Ranking

TL;DR

This paper introduces a multilingual dataset and benchmarks for scalar adjective ranking, along with a new classification task for scalar adjective identification, advancing research in multilingual NLP and semantic understanding.

Contribution

It provides the first multilingual dataset for scalar adjective ranking and establishes baseline results using contextual language models, also proposing a new scalar adjective classification task.

Findings

Baseline performance on the multilingual dataset established.

Contextual language models can distinguish scalar from relational adjectives.

New binary classification task for scalar adjective identification introduced.

Abstract

The intensity relationship that holds between scalar adjectives (e.g., nice < great < wonderful) is highly relevant for natural language inference and common-sense reasoning. Previous research on scalar adjective ranking has focused on English, mainly due to the availability of datasets for evaluation. We introduce a new multilingual dataset in order to promote research on scalar adjectives in new languages. We perform a series of experiments and set performance baselines on this dataset, using monolingual and multilingual contextual language models. Additionally, we introduce a new binary classification task for English scalar adjective identification which examines the models' ability to distinguish scalar from relational adjectives. We probe contextualised representations and report baseline results for future comparison on this task.