Data Swarms: Optimizable Generation of Synthetic Evaluation Data

TL;DR

Data Swarms introduces an optimization algorithm for generating synthetic evaluation data that improves the difficulty and robustness of evaluations for large language models, with co-evolving adversarial variants enhancing model training.

Contribution

The paper presents a novel swarm-based optimization method for synthetic data generation, including an adversarial extension for co-evolving data and models, advancing evaluation and training of LLMs.

Findings

Outperforms eight baseline data generation methods across five objectives

Generates more challenging and diverse evaluation data

Enhances model robustness and generalization to unseen LLMs

Abstract

We propose Data Swarms, an algorithm to optimize the generation of synthetic evaluation data and advance quantitative desiderata of LLM evaluation. We first train a swarm of initial data generators using existing data, and define various evaluation objectives to reflect the desired properties of evaluation (e.g., generate more difficult problems for the evaluated models) and quantitatively evaluate data generators. We then employ particle swarm optimization to optimize the swarm of data generators, where they collaboratively search through the model parameter space to find new generators that advance these objectives. We further extend it to Adversarial Swarms, where the data generator swarm generates harder data while the test taker model swarm learns from such data, co-evolving dynamically for better data and models simultaneously. Extensive experiments demonstrate that Data Swarms outperforms eight data generation baselines across five evaluation objectives, while Adversarial Swarms produce more robust learning of synthetic data and stronger generalization. Further analysis reveals that Data Swarms successfully optimizes compositions of multiple evaluation objectives and generalizes to new off-the-shelf LLMs, unseen at optimization time.