EDCO: Dynamic Curriculum Orchestration for Domain-specific Large Language Model Fine-tuning

TL;DR

EDCO introduces a dynamic curriculum learning framework for domain-specific LLM fine-tuning that adaptively selects high-entropy samples, improving training efficiency and model performance across multiple domains.

Contribution

This paper presents EDCO, a novel dynamic curriculum orchestration method based on inference entropy, which adapts sample selection during fine-tuning, unlike static curricula in prior work.

Findings

EDCO outperforms traditional static curriculum strategies in multiple domains.

Efficient entropy estimation reduces computational time by 83.5%.

EDCO enhances long-term reasoning in LLMs during fine-tuning.

Abstract

Domain-specific large language models (LLMs), typically developed by fine-tuning a pre-trained general-purpose LLM on specialized datasets, represent a significant advancement in applied AI. A common strategy in LLM fine-tuning is curriculum learning, which pre-orders training samples based on metrics like difficulty to improve learning efficiency compared to a random sampling strategy. However, most existing methods for LLM fine-tuning rely on a static curriculum, designed prior to training, which lacks adaptability to the model's evolving needs during fine-tuning. To address this, we propose EDCO, a novel framework based on two key concepts: inference entropy and dynamic curriculum orchestration. Inspired by recent findings that maintaining high answer entropy benefits long-term reasoning gains, EDCO prioritizes samples with high inference entropy in a continuously adapted curriculum. EDCO integrates three core components: an efficient entropy estimator that uses prefix tokens to approximate full-sequence entropy, an entropy-based curriculum generator that selects data points with the highest inference entropy, and an LLM trainer that optimizes the model on the selected curriculum. Comprehensive experiments in communication, medicine and law domains, EDCO outperforms traditional curriculum strategies for fine-tuning Qwen3-4B and Llama3.2-3B models under supervised and reinforcement learning settings. Furthermore, the proposed efficient entropy estimation reduces computational time by 83.5% while maintaining high accuracy.