Scalable Frame-based Construction of Sociocultural NormBases for Socially-Aware Dialogues

TL;DR

This paper presents a scalable method using Large Language Models to construct a comprehensive Chinese Sociocultural NormBase for socially-aware dialogues, leveraging synthetic data and demonstrating its effectiveness in downstream tasks.

Contribution

It introduces a novel approach for building Sociocultural NormBases with LLMs, utilizing synthetic data and contextual frames to improve quality and applicability in dialogue systems.

Findings

Synthetic data yields comparable norm quality to real data.

Frame annotations improve the quality of extracted norms.

SCNs enhance downstream dialogue reasoning tasks.

Abstract

Sociocultural norms serve as guiding principles for personal conduct in social interactions, emphasizing respect, cooperation, and appropriate behavior, which is able to benefit tasks including conversational information retrieval, contextual information retrieval and retrieval-enhanced machine learning. We propose a scalable approach for constructing a Sociocultural Norm (SCN) Base using Large Language Models (LLMs) for socially aware dialogues. We construct a comprehensive and publicly accessible Chinese Sociocultural NormBase. Our approach utilizes socially aware dialogues, enriched with contextual frames, as the primary data source to constrain the generating process and reduce the hallucinations. This enables extracting of high-quality and nuanced natural-language norm statements, leveraging the pragmatic implications of utterances with respect to the situation. As real dialogue annotated with gold frames are not readily available, we propose using synthetic data. Our empirical results show: (i) the quality of the SCNs derived from synthetic data is comparable to that from real dialogues annotated with gold frames, and (ii) the quality of the SCNs extracted from real data, annotated with either silver (predicted) or gold frames, surpasses that without the frame annotations. We further show the effectiveness of the extracted SCNs in a RAG-based (Retrieval-Augmented Generation) model to reason about multiple downstream dialogue tasks.