AQuilt: Weaving Logic and Self-Inspection into Low-Cost, High-Relevance Data Synthesis for Specialist LLMs

TL;DR

AQuilt is a cost-effective framework that enhances specialized LLM performance by generating high-relevance instruction data through logic and self-inspection, reducing costs while maintaining quality.

Contribution

It introduces a novel data synthesis approach incorporating logic and inspection, enabling high-quality, task-specific data generation with significantly lower costs.

Findings

Achieves comparable performance to DeepSeek-V3 at 17% of the cost.

Generates data with higher relevance to downstream tasks.

Constructed a 703k example dataset for training.

Abstract

Despite the impressive performance of large language models (LLMs) in general domains, they often underperform in specialized domains. Existing approaches typically rely on data synthesis methods and yield promising results by using unlabeled data to capture domain-specific features. However, these methods either incur high computational costs or suffer from performance limitations, while also demonstrating insufficient generalization across different tasks. To address these challenges, we propose AQuilt, a framework for constructing instruction-tuning data for any specialized domains from corresponding unlabeled data, including Answer, Question, Unlabeled data, Inspection, Logic, and Task type. By incorporating logic and inspection, we encourage reasoning processes and self-inspection to enhance model performance. Moreover, customizable task instructions enable high-quality data generation for any task. As a result, we construct a dataset of 703k examples to train a powerful data synthesis model. Experiments show that AQuilt is comparable to DeepSeek-V3 while utilizing just 17% of the production cost. Further analysis demonstrates that our generated data exhibits higher relevance to downstream tasks. Source code, models, and scripts are available at https://github.com/Krueske/AQuilt.