VEPO: Variable Entropy Policy Optimization for Low-Resource Language Foundation Models

TL;DR

VEPO introduces a reinforcement learning framework with variable entropy control to improve low-resource language models by enhancing tokenization and translation quality, addressing data imbalance and subword segmentation issues.

Contribution

The paper presents VEPO, a novel reinforcement learning method with variable entropy that enforces linguistic constraints and balances fidelity and naturalness in low-resource language modeling.

Findings

Significant improvements in tokenization efficiency.

Enhanced translation quality for underrepresented languages.

Bridging performance gaps in low-resource language tasks.

Abstract

Large language models frequently exhibit suboptimal performance on low resource languages, primarily due to inefficient subword segmentation and systemic training data imbalances. In this paper, we propose Variable Entropy Policy Optimization (VEPO), which leverages Reinforcement Learning with Verifiable Rewards to incorporate deterministic structural constraints into the policy alignment process. This framework ensures prescribed sequence length, robust format consistency, and rigorous linguistic well formedness, all enforced during training. Central to our approach is a variable entropy mechanism that enables the model to dynamically calibrate the equilibrium between literal fidelity and semantic naturalness by modulating the exploration exploitation manifold. By integrating entropy tempered advantage estimation with asymmetric clipping, VEPO sustains robust exploration while mitigating policy collapse. Empirical evaluations across 90 FLORES-200, COMET-22, chrF directions demonstrate that VEPO yields substantial improvements in both tokenization efficiency and translation quality, bridging the performance gap for underrepresented languages.