WILDCHAT-50M: A Deep Dive Into the Role of Synthetic Data in Post-Training

TL;DR

This paper introduces WILDCHAT-50M, the largest public chat dataset with responses from over 50 models, enabling large-scale analysis of synthetic data's role in language model post-training and demonstrating its effectiveness in improving model performance.

Contribution

The paper presents WILDCHAT-50M, a comprehensive dataset for analyzing synthetic data in LLM post-training, and develops RE-WILD, a superior SFT mixture using this dataset.

Findings

WILDCHAT-50M includes responses from 50+ models, enhancing analysis capabilities.

RE-WILD outperforms Tulu-3 SFT with fewer samples.

The dataset and code are publicly available for research use.

Abstract

Language model (LLM) post-training, from DPO to distillation, can refine behaviors and unlock new skills, but the open science supporting these post-training techniques is still in its infancy. One limiting factor has been the difficulty of conducting large-scale comparative analyses of synthetic data generating models and LLM judges. To close this gap, we introduce WILDCHAT-50M, the largest public chat dataset to date. We extend the existing WildChat dataset to include responses not only from GPT, but from over 50 different open-weight models, ranging in size from 0.5B to 104B parameters. We conduct an extensive comparative analysis and demonstrate the potential of this dataset by creating RE-WILD, our own public SFT mix, which outperforms the recent Tulu-3 SFT mixture from Allen AI with only 40% as many samples. Our dataset, samples and code are available at https://github.com/penfever/wildchat-50m.