# TCIA: A Task-Centric Instruction Augmentation Method for Instruction Finetuning

## TL;DR

TCIA is a novel instruction augmentation framework that enhances large language models' performance on specific tasks by expanding instructions while maintaining diversity and task relevance, leading to significant improvements in real-world applications.

## Contribution

Introduces Task Centric Instruction Augmentation (TCIA), a method that systematically expands instructions for better task alignment and generalization in instruction tuning.

## Key findings

- Improves open-source LLMs' performance by 8.7% on average across four applications.
- Maintains general instruction-following ability while optimizing for specific tasks.
- Outperforms some closed-source models in task-specific scenarios.

## Abstract

Diverse instruction data is vital for effective instruction tuning of large language models, as it enables the model to generalize across different types of inputs . Building such diversified instruction dataset is an essential step in this process. Existing approaches often leverage large language models to automatically explore and generate diverse instructions, ensuring both data diversity and quality. However, they tend to overlook an important factor in real-world applications: on-task relevance. In practice, only a few real-world applications require a truly general-purpose model; most benefit from task-specific knowledge tailored to their particular use case. Therefore, it is vital to develop instruction augmentation methods that not only maintain diversity but are also optimized for specific, real-world scenarios. We thus introduce Task Centric Instruction Augmentation (TCIA), a framework that systematically expands instructions while preserving both diversity and task alignment. By representing instructions in a discrete query-constraints space, TCIA creates a rich set of task-relevant instructions and enables models to generalize to these task-specific instructions without sacrificing overall performance. Experiments show that TCIA improves open-source LLMs' performance by an average of 8.7% across four real-world, task-specific applications, and in some cases outperforming leading closed-source models. These improvements do not compromise general instruction-following ability, making TCIA a scalable and efficient solution for adapting LLMs to real-world, task-focused applications.