AdaCultureSafe: Adaptive Cultural Safety Grounded by Cultural Knowledge in Large Language Models

TL;DR

This paper introduces AdaCultureSafe, a dataset and framework for improving cultural safety in LLMs by grounding responses in authoritative cultural knowledge, addressing challenges of cultural diversity and subtlety.

Contribution

It presents a novel dataset and a knowledge-grounded approach to enhance cultural safety in LLMs, bridging the gap between cultural knowledge and safety.

Findings

No significant correlation between cultural safety and knowledge proficiency in LLMs.

The proposed method significantly improves cultural safety in LLM responses.

A new dataset with 4.8K cultural descriptions and 48K queries was created.

Abstract

With the widespread adoption of Large Language Models (LLMs), respecting indigenous cultures becomes essential for models' culturally safety and responsible global applications. Existing studies separately consider cultural safety and cultural knowledge and neglect that the former should be grounded by the latter. This severely prevents LLMs from yielding culture-specific respectful responses. Consequently, adaptive cultural safety remains a formidable task. In this work, we propose to jointly model cultural safety and knowledge. First and foremost, cultural-safety and knowledge-paired data serve as the key prerequisite to conduct this research. However, the cultural diversity across regions and the subtlety of cultural differences pose significant challenges to the creation of such paired evaluation data. To address this issue, we propose a novel framework that integrates authoritative cultural knowledge descriptions curation, LLM-automated query generation, and heavy manual verification. Accordingly, we obtain a dataset named AdaCultureSafe containing 4.8K manually decomposed fine-grained cultural descriptions and the corresponding 48K manually verified safety- and knowledge-oriented queries. Upon the constructed dataset, we evaluate three families of popular LLMs on their cultural safety and knowledge proficiency, via which we make a critical discovery: no significant correlation exists between their cultural safety and knowledge proficiency. We then delve into the utility-related neuron activations within LLMs to investigate the potential cause of the absence of correlation, which can be attributed to the difference of the objectives of pre-training and post-alignment. We finally present a knowledge-grounded method, which significantly enhances cultural safety by enforcing the integration of knowledge into the LLM response generation process.