WizardCoder: Empowering Code Large Language Models with Evol-Instruct

TL;DR

WizardCoder enhances code large language models with instruction fine-tuning, significantly improving their performance on code generation benchmarks and surpassing both open-source and some closed models.

Contribution

Introduces WizardCoder, a code LLM fine-tuned with Evol-Instruct, achieving state-of-the-art results on multiple code benchmarks.

Findings

Outperforms all open-source Code LLMs on key benchmarks.

Surpasses large closed LLMs like Claude and Bard on HumanEval.

Code, models, and data are publicly available.

Abstract

Code Large Language Models (Code LLMs), such as StarCoder, have demonstrated exceptional performance in code-related tasks. However, most existing models are solely pre-trained on extensive raw code data without instruction fine-tuning. In this paper, we introduce WizardCoder, which empowers Code LLMs with complex instruction fine-tuning, by adapting the Evol-Instruct method to the domain of code. Through comprehensive experiments on four prominent code generation benchmarks, namely HumanEval, HumanEval+, MBPP, and DS-1000, we unveil the exceptional capabilities of our model. It surpasses all other open-source Code LLMs by a substantial margin. Moreover, our model even outperforms the largest closed LLMs, Anthropic's Claude and Google's Bard, on HumanEval and HumanEval+. Our code, model weights, and data are public at https://github.com/nlpxucan/WizardLM