Persian-Phi: Efficient Cross-Lingual Adaptation of Compact LLMs via Curriculum Learning

TL;DR

Persian-Phi demonstrates that a compact 3.8B parameter model can be effectively adapted to Persian using a resource-efficient curriculum learning pipeline, enabling high performance in low-resource language settings.

Contribution

This work introduces a novel curriculum learning approach for adapting large language models to low-resource languages with minimal resources.

Findings

Achieves competitive results on Open Persian LLM Leaderboard

Employs a unique bilingual warm-up stage for embedding alignment

Validates scalable adaptation of compact models to underrepresented languages

Abstract

The democratization of AI is currently hindered by the immense computational costs required to train Large Language Models (LLMs) for low-resource languages. This paper presents Persian-Phi, a 3.8B parameter model that challenges the assumption that robust multilingual capabilities require massive model sizes or multilingual baselines. We demonstrate how Microsoft Phi-3 Mini -- originally a monolingual English model -- can be effectively adapted to Persian through a novel, resource-efficient curriculum learning pipeline. Our approach employs a unique "warm-up" stage using bilingual narratives (Tiny Stories) to align embeddings prior to heavy training, followed by continual pretraining and instruction tuning via Parameter-Efficient Fine-Tuning (PEFT). Despite its compact size, Persian-Phi achieves competitive results on Open Persian LLM Leaderboard in HuggingFace. Our findings provide a validated, scalable framework for extending the reach of state-of-the-art LLMs to underrepresented languages with minimal hardware resources. The Persian-Phi model is publicly available at https://huggingface.co/amirakhlaghiqqq/PersianPhi.