CREST: Universal Safety Guardrails Through Cluster-Guided Cross-Lingual Transfer

TL;DR

CREST is a multilingual safety classification model supporting 100 languages with only 0.5B parameters, using cluster-based transfer from 13 high-resource languages to generalize safety guardrails globally.

Contribution

Introducing CREST, a parameter-efficient, cross-lingual safety model that outperforms similar-sized models and generalizes to low-resource languages using cluster-guided transfer.

Findings

CREST supports 100 languages with only 0.5B parameters.

It outperforms existing safety guardrails of similar scale.

It achieves competitive results against larger models with 2.5B+ parameters.

Abstract

Ensuring content safety in large language models (LLMs) is essential for their deployment in real-world applications. However, existing safety guardrails are predominantly tailored for high-resource languages, leaving a significant portion of the world's population underrepresented who communicate in low-resource languages. To address this, we introduce CREST (CRoss-lingual Efficient Safety Transfer), a parameter-efficient multilingual safety classification model that supports 100 languages with only 0.5B parameters. By training on a strategically chosen subset of only 13 high-resource languages, our model utilizes cluster-based cross-lingual transfer from a few to 100 languages, enabling effective generalization to both unseen high-resource and low-resource languages. This approach addresses the challenge of limited training data in low-resource settings. We conduct comprehensive evaluations across six safety benchmarks to demonstrate that CREST outperforms existing state-of-the-art guardrails of comparable scale and achieves competitive results against models with significantly larger parameter counts (2.5B parameters and above). Our findings highlight the limitations of language-specific guardrails and underscore the importance of developing universal, language-agnostic safety systems that can scale effectively to serve global populations.