Self-Alignment: Improving Alignment of Cultural Values in LLMs via In-Context Learning

TL;DR

This paper introduces a simple in-context learning method that leverages human survey data to enhance the cultural value alignment of large language models across multiple languages and diverse countries.

Contribution

It proposes an inexpensive, effective approach to improve LLMs' cultural value alignment using in-context learning combined with survey data, applicable across languages.

Findings

Improved cultural value alignment in 5 LLMs including multilingual models.

Effective in both English and non-English languages.

Enhances alignment with diverse cultural values from various countries.

Abstract

Improving the alignment of Large Language Models (LLMs) with respect to the cultural values that they encode has become an increasingly important topic. In this work, we study whether we can exploit existing knowledge about cultural values at inference time to adjust model responses to cultural value probes. We present a simple and inexpensive method that uses a combination of in-context learning (ICL) and human survey data, and show that we can improve the alignment to cultural values across 5 models that include both English-centric and multilingual LLMs. Importantly, we show that our method could prove useful in test languages other than English and can improve alignment to the cultural values that correspond to a range of culturally diverse countries.