Convex Dataset Valuation for Post-Training

TL;DR

This paper introduces a convex dataset valuation method using kernel mean matching to optimize auxiliary dataset selection for post-training large language models under budget constraints.

Contribution

It proposes a scalable convex valuation approach that considers both dataset alignment and redundancy, outperforming existing methods in post-training LLM tasks.

Findings

Our method outperforms existing valuation baselines in diverse tasks.

It achieves stronger performance with low computational overhead.

The approach effectively balances dataset relevance and redundancy.

Abstract

Improving LLM performance on downstream tasks sometimes requires leveraging auxiliary datasets during post-training. In practice, however, developers face constraints on compute, labeling, and licensing costs that preclude using all available data, necessitating principled dataset-level selection. These constraints are increasingly shaped by dataset marketplaces, where data acquisition is governed by budgets and negotiation. We study dataset valuation as a subset selection problem during LLM post-training. Our goal is to identify and weight auxiliary datasets so as to maximize target task performance given constrained budgets. We first show that commonly used gradient alignment scores provide a reasonable yet incomplete valuation signal, as they ignore redundancy among datasets. To address this, we propose a scalable convex dataset-level valuation method based on kernel mean matching (KMM) in gradient space, which jointly accounts for alignment with the target task and redundancy across auxiliary datasets. Through extensive experiments across diverse post-training settings and tasks, we show that our approach consistently outperforms existing valuation baselines, achieving stronger performance with low computational overhead. Our results position dataset valuation as a practical decision tool for post-training data selection in market-constrained large language model settings. The code is available at https://github.com/uiuctml/convex_data_valuation.