AceGPT, Localizing Large Language Models in Arabic

TL;DR

AceGPT is a culturally sensitive Arabic language model developed through targeted pre-training, fine-tuning, and reinforcement learning, achieving state-of-the-art performance on Arabic benchmarks.

Contribution

This work introduces AceGPT, a novel Arabic LLM that incorporates cultural and value alignment through specialized training and reinforcement learning techniques.

Findings

Achieves state-of-the-art results on Arabic benchmarks

Demonstrates improved cultural sensitivity and value alignment

Provides open-source code, data, and models

Abstract

This paper is devoted to the development of a localized Large Language Model (LLM) specifically for Arabic, a language imbued with unique cultural characteristics inadequately addressed by current mainstream models. Significant concerns emerge when addressing cultural sensitivity and local values. To address this, the paper proposes a comprehensive solution that includes further pre-training with Arabic texts, Supervised Fine-Tuning (SFT) utilizing native Arabic instructions, and GPT-4 responses in Arabic, alongside Reinforcement Learning with AI Feedback (RLAIF) employing a reward model attuned to local culture and values. The goal is to cultivate culturally cognizant and value-aligned Arabic LLMs capable of accommodating the diverse, application-specific needs of Arabic-speaking communities. Comprehensive evaluations reveal that the resulting model, dubbed `AceGPT', sets the state-of-the-art standard for open Arabic LLMs across various benchmarks. Codes, data, and models are in https://github.com/FreedomIntelligence/AceGPT.