Linguistic and Argument Diversity in Synthetic Data for Function-Calling Agents

TL;DR

This paper introduces a method for generating diverse synthetic datasets for function-calling agents, enhancing linguistic and argument diversity without manual rules, leading to improved model performance especially out-of-distribution.

Contribution

It proposes an optimization-based approach for synthetic data generation that improves diversity and model accuracy without relying on handcrafted rules.

Findings

Outperforms state-of-the-art data generation methods in diversity metrics

Achieves 7.4% higher accuracy on BFCL benchmark

Maintains comparable correctness while increasing diversity

Abstract

The construction of function calling agents has emerged as a promising avenue for extending model capabilities. A major challenge for this task is obtaining high quality diverse data for training. Prior work emphasizes diversity in functions, invocation patterns, and interaction turns, yet linguistic diversity of requests and coverage of arguments (e.g., \texttt{city\_name}, \texttt{stock\_ticker}) remain underexplored. We propose a method that generates synthetic datasets via optimizing general-purpose diversity metrics across both queries and arguments, without relying on hand-crafted rules or taxonomies, making it robust to different usecases. We demonstrate the effectiveness of our technique via both intrinsic and extrinsic testing, comparing it to SoTA data generation methods. We show a superiority over baselines in terms of diversity, while keeping comparable correctness. Additionally, when used as a training set, the model resulting from our dataset exhibits superior performance compared to analogous models based on the baseline data generation methods in out-of-distribution performance. In particular, we achieve an $7.4\%$ increase in accuracy on the BFCL benchmark compared to similar counterparts.