TyDiP: A Dataset for Politeness Classification in Nine Typologically Diverse Languages

TL;DR

This paper introduces TyDiP, a multilingual politeness dataset across nine diverse languages, evaluates model performance in identifying politeness, and explores cross-lingual strategies and the relationship between politeness and formality.

Contribution

The paper presents TyDiP, a novel dataset for politeness classification in nine languages, and analyzes multilingual model transferability and politeness-formality relationships.

Findings

Multilingual models show robust zero-shot politeness detection.

Models significantly underperform compared to human accuracy.

English politeness strategies can be mapped across languages.

Abstract

We study politeness phenomena in nine typologically diverse languages. Politeness is an important facet of communication and is sometimes argued to be cultural-specific, yet existing computational linguistic study is limited to English. We create TyDiP, a dataset containing three-way politeness annotations for 500 examples in each language, totaling 4.5K examples. We evaluate how well multilingual models can identify politeness levels -- they show a fairly robust zero-shot transfer ability, yet fall short of estimated human accuracy significantly. We further study mapping the English politeness strategy lexicon into nine languages via automatic translation and lexicon induction, analyzing whether each strategy's impact stays consistent across languages. Lastly, we empirically study the complicated relationship between formality and politeness through transfer experiments. We hope our dataset will support various research questions and applications, from evaluating multilingual models to constructing polite multilingual agents.