Instruction Tuning with Human Curriculum

TL;DR

This paper introduces Curriculum Instruction Tuning, a method that organizes instruction data by difficulty to improve model performance across multiple benchmarks without extra computational cost.

Contribution

It presents a systematic curriculum-based data generation pipeline and demonstrates significant performance gains through curriculum ordering in instruction tuning.

Findings

Performance improvements of +4.76 on TruthfulQA and +2.98 on MMLU

Consistent benefits across nine benchmarks

No additional computational expenses required

Abstract

In this work, we (1) introduce Curriculum Instruction Tuning, (2) explore the potential advantages of employing diverse curriculum strategies, and (3) delineate a synthetic instruction-response generation framework that complements our theoretical approach. Distinct from the existing instruction tuning dataset, our generation pipeline is systematically structured to emulate the sequential and orderly characteristic of human learning. Additionally, we describe a methodology for generating instruction-response datasets that extensively span the various stages of human education, from middle school through the graduate level, utilizing educational subject catalogs. Before training, we meticulously organize the instruction data to ensure that questions escalate in difficulty regarding (A) the subject matter and (B) the intricacy of the instructions. The findings of our study reveal that substantial improvements in performance can be achieved through the mere application of curriculum ordering to instruction data (achieving gains of +4.76 on TruthfulQA, +2.98 on MMLU, +2.8 on OpenbookQA, and +1.28 on ARC-hard) compared to random shuffling. This enhancement is achieved without incurring additional computational expenses. Through comprehensive experimentation, we observe that the advantages of our proposed method are consistently evident across nine benchmarks.