Learning What Matters: Probabilistic Task Selection via Mutual Information for Model Finetuning

TL;DR

This paper introduces TASKPGM, a probabilistic framework for optimizing task mixtures in LLM finetuning, balancing diversity and representativeness to improve performance and interpretability.

Contribution

We propose a scalable, principled method for automatic task mixture selection using mutual information and MRF modeling, with theoretical guarantees and empirical validation.

Findings

Consistent performance improvements on Llama 2 and Mistral models.

Provides interpretable insights into task influence and mixture composition.

Achieves a closed-form solution for mixture optimization under simplex constraints.

Abstract

The performance of finetuned large language models (LLMs) hinges critically on the composition of the training mixture. However, selecting an optimal blend of task datasets remains a largely manual, heuristic driven process, with practitioners often relying on uniform or size based sampling strategies. We introduce TASKPGM, a principled and scalable framework for mixture optimization that selects continuous task proportions by minimizing an energy function over a Markov Random Field (MRF). Task relationships are modeled using behavioral divergences such as Jensen Shannon Divergence and Pointwise Mutual Information computed from the predictive distributions of single task finetuned models. Our method yields a closed form solution under simplex constraints and provably balances representativeness and diversity among tasks. We provide theoretical guarantees, including weak submodularity for budgeted variants, and demonstrate consistent empirical improvements on Llama 2 and Mistral across evaluation suites such as MMLU and BIGBench. Beyond performance, TASKPGM offers interpretable insights into task influence and mixture composition, making it a powerful tool for efficient and robust LLM finetuning.