Subset Selection for Fine-Tuning: A Utility-Diversity Balanced Approach for Mathematical Domain Adaptation

TL;DR

This paper introduces a utility-diversity balanced subset selection method for fine-tuning large language models on mathematical data, reducing training costs while maintaining high performance.

Contribution

It presents a novel subset selection approach combining utility and diversity metrics to efficiently fine-tune LLMs for mathematical domains.

Findings

Achieves near-full dataset performance with fewer training examples

Reduces computational cost and training time significantly

Outperforms baseline subset selection methods

Abstract

We propose a refined approach to efficiently fine-tune large language models (LLMs) on specific domains like the mathematical domain by employing a budgeted subset selection method. Our approach combines utility and diversity metrics to select the most informative and representative training examples. The final goal is to achieve near-full dataset performance with meticulously selected data points from the entire dataset while significantly reducing computational cost and training time and achieving competitive performance as the full dataset. The utility metric incorporates both perplexity and Chain-of-Thought (CoT) loss to identify challenging examples that contribute most to model learning, while the diversity metric ensures broad coverage across mathematical subdomains. We evaluate our method on LLaMA-3 8B and Phi-3 models, comparing against several baseline approaches, including random selection, diversity-based sampling, and existing state-of-the-art subset selection techniques.