AKD : Adversarial Knowledge Distillation For Large Language Models Alignment on Coding tasks

TL;DR

This paper introduces AKD, an adversarial knowledge distillation method that improves the robustness, reliability, and efficiency of large language models for code generation by using synthetic adversarial datasets.

Contribution

The paper presents a novel adversarial knowledge distillation framework that enhances Code-LLMs' capabilities and robustness using synthetic adversarial data, addressing data scarcity and model scaling issues.

Findings

AKD improves model robustness and reliability.

Synthetic adversarial datasets enhance reasoning capabilities.

Smaller models retain performance comparable to larger ones.

Abstract

The widespread adoption of Large Language Models (LLMs) for code generation, exemplified by GitHub Copilot\footnote{A coding extension powered by a Code-LLM to assist in code completion tasks} surpassing a million users, highlights the transformative potential of these tools in improving developer productivity. However, this rapid growth also underscores critical concerns regarding the quality, safety, and reliability of the code they generate. As Code-LLMs evolve, they face significant challenges, including the diminishing returns of model scaling and the scarcity of new, high-quality training data. To address these issues, this paper introduces Adversarial Knowledge Distillation (AKD), a novel approach that leverages adversarially generated synthetic datasets to distill the capabilities of larger models into smaller, more efficient ones. By systematically stress-testing and refining the reasoning capabilities of Code-LLMs, AKD provides a framework for enhancing model robustness, reliability, and security while improving their parameter-efficiency. We believe this work represents a critical step toward ensuring dependable automated code generation within the constraints of existing data and the cost-efficiency of model execution.