Montessori-Instruct: Generate Influential Training Data Tailored for Student Learning
Xiaochuan Li, Zichun Yu, Chenyan Xiong

TL;DR
Montessori-Instruct is a novel data synthesis framework that tailors synthetic training data to enhance student language model learning by leveraging local data influence and direct preference optimization, significantly improving performance.
Contribution
It introduces a new method for generating student-specific synthetic data using local influence and DPO, outperforming standard methods and stronger teachers.
Findings
Outperforms standard synthesis methods by 18.35% and 46.24% on benchmarks.
Surpasses data from a stronger teacher model, GPT-4o.
Demonstrates robustness across different student models.
Abstract
Synthetic data has been widely used to train large language models, but their generative nature inevitably introduces noisy, non-informative, and misleading learning signals. In this paper, we propose Montessori-Instruct, a novel data synthesis framework that tailors the data synthesis ability of the teacher language model toward the student language model's learning process. Specifically, we utilize local data influence of synthetic training data points on students to characterize students' learning preferences. Then, we train the teacher model with Direct Preference Optimization (DPO) to generate synthetic data tailored toward student learning preferences. Experiments with Llama3-8B-Instruct (teacher) and Llama3-8B (student) on Alpaca Eval and MT-Bench demonstrate that Montessori-Instruct significantly outperforms standard synthesis methods by 18.35\% and 46.24\% relatively. Our…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsEducation Methods and Practices
