FABRIC: Framework for Agent-Based Realistic Intelligence Creation

TL;DR

This paper introduces FABRIC, a scalable framework that synthesizes detailed agent interaction data using only LLMs, facilitating the development of more capable and robust agent-based language models without human supervision.

Contribution

FABRIC provides a modular, constraint-driven pipeline for generating comprehensive, high-quality agentic datasets entirely with LLMs, eliminating the need for costly human annotation.

Findings

Successfully generates multi-task, multi-turn interaction records

Ensures data quality through validation and filtering

Enables scalable creation of agentic datasets

Abstract

Large language models (LLMs) are increasingly deployed as agents, expected to decompose goals, invoke tools, and verify results in dynamic environments. Realizing these capabilities requires access to agentic data-structured interaction records that couple user intents with tool specifications, argument-grounded calls, and verifiable execution traces. However, collecting such data from human annotators is costly, time-consuming, and difficult to scale. We present a unified framework for synthesizing agentic data using only LLMs, without any human-in-the-loop supervision. This framework decomposes generation into modular pipelines that produce complete interaction records spanning task specifications, tool definitions, policy pseudocode, natural language exchanges, and execution traces. Records conform to strict syntactic and semantic constraints, ensuring machine-parseability and faithful alignment across inputs, outputs, and tool calls. Beyond single tasks, there is support for both multi-task and multi-turn agent interactions, enabling the construction of datasets that reflect the full spectrum of tool-use competencies. To ensure quality and consistency, the framework integrates constrained generation formats, JSON-schema validation, and judge-based filtering. This paper formalizes the schema for agentic records, details the prompt design principles that guide generation, and introduces scalable pipelines for high-quality synthetic data. By providing a reproducible, LLM-only alternative to manual collection, hence advancing the development of agentic LLMs capable of robust tool use.