Language verY Rare for All
Ibrahim Merad, Amos Wolf, Ziad Mazzawi, Yannick L\'eo

TL;DR
This paper introduces LYRA, a novel approach combining open LLM fine-tuning, retrieval-augmented generation, and transfer learning to improve translation of rare languages like Monégasque using only a single GPU.
Contribution
LYRA is a new method that enhances rare language translation by integrating fine-tuning, retrieval, and transfer learning, enabling effective single-GPU training.
Findings
LYRA outperforms existing models in rare language translation.
LYRA matches state-of-the-art encoder-decoder models.
The approach is effective for low-resource language pairs.
Abstract
In the quest to overcome language barriers, encoder-decoder models like NLLB have expanded machine translation to rare languages, with some models (e.g., NLLB 1.3B) even trainable on a single GPU. While general-purpose LLMs perform well in translation, open LLMs prove highly competitive when fine-tuned for specific tasks involving unknown corpora. We introduce LYRA (Language verY Rare for All), a novel approach that combines open LLM fine-tuning, retrieval-augmented generation (RAG), and transfer learning from related high-resource languages. This study is exclusively focused on single-GPU training to facilitate ease of adoption. Our study focuses on two-way translation between French and Mon\'egasque, a rare language unsupported by existing translation tools due to limited corpus availability. Our results demonstrate LYRA's effectiveness, frequently surpassing and consistently matching…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNatural Language Processing Techniques · Topic Modeling · Generative Adversarial Networks and Image Synthesis
