Fixing It in Post: A Comparative Study of LLM Post-Training Data Quality and Model Performance

TL;DR

This paper systematically compares two open post-training datasets for large language models, analyzes their quality, and creates a new optimized dataset that improves model performance efficiently.

Contribution

It provides the first detailed comparison of open post-training datasets, introduces a new curation method, and releases resources for future research.

Findings

TuluTalk dataset outperforms source datasets on key benchmarks.

Structural and quality differences identified between Tulu-3-SFT-Mix and SmolTalk.

Curated dataset achieves similar or better performance with 14% fewer samples.

Abstract

Recent work on large language models (LLMs) has increasingly focused on post-training and alignment with datasets curated to enhance instruction following, world knowledge, and specialized skills. However, most post-training datasets used in leading open- and closed-source LLMs remain inaccessible to the public, with limited information about their construction process. This lack of transparency has motivated the recent development of open-source post-training corpora. While training on these open alternatives can yield performance comparable to that of leading models, systematic comparisons remain challenging due to the significant computational cost of conducting them rigorously at scale, and are therefore largely absent. As a result, it remains unclear how specific samples, task types, or curation strategies influence downstream performance when assessing data quality. In this work, we conduct the first comprehensive side-by-side analysis of two prominent open post-training datasets: Tulu-3-SFT-Mix and SmolTalk. Using the Magpie framework, we annotate each sample with detailed quality metrics, including turn structure (single-turn vs. multi-turn), task category, input quality, and response quality, and we derive statistics that reveal structural and qualitative similarities and differences between the two datasets. Based on these insights, we design a principled curation recipe that produces a new data mixture, TuluTalk, which contains 14% fewer samples than either source dataset while matching or exceeding their performance on key benchmarks. Our findings offer actionable insights for constructing more effective post-training datasets that improve model performance within practical resource limits. To support future research, we publicly release both the annotated source datasets and our curated TuluTalk mixture.