CulFiT: A Fine-grained Cultural-aware LLM Training Paradigm via Multilingual Critique Data Synthesis

TL;DR

CulFiT is a new training paradigm for LLMs that uses multilingual critique data and fine-grained rewards to improve cultural sensitivity, inclusivity, and reduce biases across diverse languages and cultures.

Contribution

It introduces a novel culturally-aware training method utilizing multilingual data synthesis and fine-grained reward modeling, along with a new dataset for evaluating cultural responses.

Findings

Achieves state-of-the-art performance in cultural alignment.

Improves model reasoning and inclusivity across cultures.

Demonstrates effectiveness on multiple benchmarks.

Abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities across various tasks, yet they often exhibit a specific cultural biases, neglecting the values and linguistic diversity of low-resource regions. This cultural bias not only undermines universal equality, but also risks reinforcing stereotypes and perpetuating discrimination. To address this, we propose CulFiT, a novel culturally-aware training paradigm that leverages multilingual data and fine-grained reward modeling to enhance cultural sensitivity and inclusivity. Our approach synthesizes diverse cultural-related questions, constructs critique data in culturally relevant languages, and employs fine-grained rewards to decompose cultural texts into verifiable knowledge units for interpretable evaluation. We also introduce GlobalCultureQA, a multilingual open-ended question-answering dataset designed to evaluate culturally-aware responses in a global context. Extensive experiments on three existing benchmarks and our GlobalCultureQA demonstrate that CulFiT achieves state-of-the-art open-source model performance in cultural alignment and general reasoning.