C-Mining: Unsupervised Discovery of Seeds for Cultural Data Synthesis via Geometric Misalignment

TL;DR

C-Mining introduces an unsupervised, geometric approach to automatically discover cultural seeds from multilingual data, improving cultural understanding in language models without manual curation.

Contribution

The paper presents a novel geometric misalignment method for quantifying and extracting cultural seeds, reducing reliance on manual curation and LLM extraction.

Findings

Achieves over 150-fold reduction in seed preparation costs.

Improves cultural reasoning by +6.03 points on CulturalBench-Hard.

Outperforms state-of-the-art baselines in cultural understanding.

Abstract

Achieving cultural alignment in Large Language Models (LLMs) increasingly depends on synthetic data generation. For such synthesis, the most vital initial step is seed curation; however, current methods lack quantifiable standards for selecting these seeds. Existing approaches rely on unscalable manual curation or bias-prone LLM extraction, treating cultural specificity as an abstract concept rather than a measurable signal. In this paper, we address this "quantification gap" by proposing C-Mining, an unsupervised framework that transforms the discovery of cultural seeds from a subjective selection process into a computable data mining formulation. Our approach exploits a novel geometric insight, leveraging the cross-lingual misalignment of cultural concepts within pre-trained embedding spaces as a quantifiable discovery signal. By systematically identifying these regions characterized by pronounced linguistic exclusivity and geometric isolation, while actively filtering out noise, C-Mining automatically extracts high-fidelity Culture Points (CPs) from raw multilingual corpora without reliance on human or LLM supervision, reducing preparation costs by more than 150-fold. We further leverage the mined knowledge to steer the synthesis of diverse instruction-tuning datasets. Extensive experiments demonstrate that this seed-centric approach significantly enhances cultural understanding and reasoning capabilities, achieving a +6.03 point improvement on CulturalBench-Hard and surpassing state-of-the-art baselines, providing a scalable, quantifiable solution for high-quality cultural data synthesis.