SyGra: A Unified Graph-Based Framework for Scalable Generation, Quality Tagging, and Management of Synthetic Data

TL;DR

SyGra is a modular, scalable framework that generates, tags, and manages high-quality synthetic dialogue data for training large language models, reducing data preparation effort.

Contribution

It introduces a configurable, dual-stage quality tagging system and a flexible schema for synthetic data tailored to SFT and DPO training methods.

Findings

Enables high-fidelity synthetic data generation at scale

Automates quality filtering with heuristic and LLM-based evaluations

Supports seamless integration into diverse LLM training workflows

Abstract

The advancement of large language models (LLMs) is critically dependent on the availability of high-quality datasets for Supervised Fine-Tuning (SFT), alignment tasks like Direct Preference Optimization (DPO), etc. In this work, we present a comprehensive synthetic data generation framework that facilitates scalable, configurable, and high-fidelity generation of synthetic data tailored for these training paradigms. Our approach employs a modular and configuration-based pipeline capable of modeling complex dialogue flows with minimal manual intervention. This framework uses a dual-stage quality tagging mechanism, combining heuristic rules and LLM-based evaluations, to automatically filter and score data extracted from OASST-formatted conversations, ensuring the curation of high-quality dialogue samples. The resulting datasets are structured under a flexible schema supporting both SFT and DPO use cases, enabling seamless integration into diverse training workflows. Together, these innovations offer a robust solution for generating and managing synthetic conversational data at scale, significantly reducing the overhead of data preparation in LLM training pipelines.