RECAST: Expanding the Boundaries of LLMs' Complex Instruction Following with Multi-Constraint Data

TL;DR

This paper introduces RECAST, a scalable framework for creating large datasets with numerous constraints to improve LLMs' ability to follow complex instructions, surpassing previous limitations.

Contribution

We propose RECAST, a novel method for synthesizing datasets with over 10 constraints per instance, significantly expanding the complexity of instruction-following benchmarks for LLMs.

Findings

Models trained on RECAST-30K show improved complex instruction following.

RECAST enables automatic verification of constraints, facilitating reinforcement learning.

Fine-tuning on RECAST-30K maintains general model capabilities.

Abstract

Large language models (LLMs) are increasingly expected to tackle complex tasks, driven by their expanding applications and users' growing proficiency in crafting sophisticated prompts. However, as the number of explicitly stated requirements increases (particularly more than 10 constraints), LLMs often struggle to accurately follow such complex instructions, which limits their applicability in complex real-world scenarios. To the best of our knowledge, existing datasets do not exceed 10 constraints per instance. To address this challenge, we propose RECAST, an efficient and scalable framework for synthesizing datasets where each example incorporates far more constraints than those in existing benchmarks, aiming to challenge and extend the boundaries of models' ability to follow complex instructions. These constraints are extracted from real-world prompt-response pairs to ensure practical relevance. Using this framework, we construct RECAST-30K, a large-scale, high-quality dataset comprising 30k instances spanning 19 constraint types. Experimental results demonstrate that models finetuned on RECAST-30K substantially improve in following complex instructions while maintaining their general capabilities without degradation. Moreover, RECAST enables automatic verification of constraint satisfaction via rule-based validators for quantitative constraints and LLM-based validators for qualitative ones; the verifiability provided by RECAST enables the design of reward functions for reinforcement learning, which further boosts model performance on complex and challenging tasks.