EasyDistill: A Comprehensive Toolkit for Effective Knowledge Distillation of Large Language Models

TL;DR

EasyDistill is a versatile, user-friendly toolkit that simplifies the application of advanced knowledge distillation techniques to large language models, facilitating research and industrial deployment.

Contribution

We introduce EasyDistill, a comprehensive toolkit with modular design, supporting various KD methods and integration into cloud platforms for LLMs.

Findings

Provides robust distilled models and datasets for diverse NLP tasks.

Enables seamless experimentation with KD strategies for LLMs.

Successfully integrated into Alibaba Cloud's PAI platform.

Abstract

In this paper, we present EasyDistill, a comprehensive toolkit designed for effective black-box and white-box knowledge distillation (KD) of large language models (LLMs). Our framework offers versatile functionalities, including data synthesis, supervised fine-tuning, ranking optimization, and reinforcement learning techniques specifically tailored for KD scenarios. The toolkit accommodates KD functionalities for both System 1 (fast, intuitive) and System 2 (slow, analytical) models. With its modular design and user-friendly interface, EasyDistill empowers researchers and industry practitioners to seamlessly experiment with and implement state-of-the-art KD strategies for LLMs. In addition, EasyDistill provides a series of robust distilled models and KD-based industrial solutions developed by us, along with the corresponding open-sourced datasets, catering to a variety of use cases. Furthermore, we describe the seamless integration of EasyDistill into Alibaba Cloud's Platform for AI (PAI). Overall, the EasyDistill toolkit makes advanced KD techniques for LLMs more accessible and impactful within the NLP community.