Self-Pluralising Culture Alignment for Large Language Models

TL;DR

This paper introduces CultureSPA, a framework enabling large language models to align with multiple cultures simultaneously, improving cultural sensitivity without sacrificing overall performance.

Contribution

It presents a novel self-pluralising approach for culture alignment in LLMs, utilizing generated questions and comparative outputs for effective fine-tuning.

Findings

Significant improvement in cultural alignment of LLMs

Enhanced performance when combined with prompt engineering

Robustness across different data qualities and quantities

Abstract

As large language models (LLMs) become increasingly accessible in many countries, it is essential to align them to serve pluralistic human values across cultures. However, pluralistic culture alignment in LLMs remain an open problem. In this paper, we propose CultureSPA, a Self-Pluralising Culture Alignment framework that allows LLMs to simultaneously align to pluralistic cultures. The framework first generates questions on various culture topics, then yields LLM outputs in response to these generated questions under both culture-aware and culture-unaware settings. By comparing culture-aware/unaware outputs, we are able to detect and collect culture-related instances. These instances are employed to fine-tune LLMs to serve pluralistic cultures in either a culture-joint or culture-specific way. Extensive experiments demonstrate that CultureSPA significantly improves the alignment of LLMs to diverse cultures without compromising general abilities. And further improvements can be achieved if CultureSPA is combined with advanced prompt engineering techniques. Comparisons between culture-joint and culture-specific tuning strategies, along with variations in data quality and quantity, illustrate the robustness of our method. We also explore the mechanisms underlying CultureSPA and the relations between different cultures it reflects.