NormDial: A Comparable Bilingual Synthetic Dialog Dataset for Modeling Social Norm Adherence and Violation

TL;DR

NormDial is a high-quality, bilingual synthetic dialogue dataset with annotations on social norm adherence and violations, enabling the study of cross-cultural social norms in conversations.

Contribution

The paper introduces NormDial, a novel synthetic dataset for modeling social norm adherence and violation in Chinese and American cultures, generated via a human-in-the-loop LLM pipeline.

Findings

High-quality dialogue dataset validated by human evaluation

Existing large language models show promising performance on social norm detection

Cross-cultural analysis reveals nuanced differences in social norm manifestation

Abstract

Social norms fundamentally shape interpersonal communication. We present NormDial, a high-quality dyadic dialogue dataset with turn-by-turn annotations of social norm adherences and violations for Chinese and American cultures. Introducing the task of social norm observance detection, our dataset is synthetically generated in both Chinese and English using a human-in-the-loop pipeline by prompting large language models with a small collection of expert-annotated social norms. We show that our generated dialogues are of high quality through human evaluation and further evaluate the performance of existing large language models on this task. Our findings point towards new directions for understanding the nuances of social norms as they manifest in conversational contexts that span across languages and cultures.