GRASP LoRA: GRPO Guided Adapter Sparsity Policy for Cross Lingual Transfer
Besher Hassan, Xiuying Chen

TL;DR
GRASP LoRA introduces a learnable sparsity policy for efficient cross-lingual transfer of large language models, reducing computational costs and improving performance without extensive grid search tuning.
Contribution
It proposes a novel adaptive sparsity control method that replaces grid search with an online learning approach for better efficiency and effectiveness in multilingual model adaptation.
Findings
Improves semantic faithfulness and content coverage in cross-lingual tasks.
Reduces runtime significantly compared to grid search methods.
Enhances answer quality in multilingual question answering.
Abstract
Parameter efficient fine tuning is a way to adapt LLMs to new languages when compute or data are limited, yet adapter pipelines usually choose a global prune ratio by grid search. This practice is computationally expensive and development set intensive, since it repeats training, freezes sparsity, and misses fractional optima. We introduce GRASP LoRA (GRPO Guided Adapter Sparsity Policy), which treats global sparsity as a learnable control variable. A GRPO controller interleaves with training, periodically probing candidate prune ratios on a small micro development set and updating a single global prune ratio online from its reward signal. It operates on merged source and target LoRA adapters on a frozen backbone and replaces grid search with one controller run that learns a prune ratio, followed by a single final merge and prune fine tuning run with pruning fixed to that ratio. On…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Multimodal Machine Learning Applications
