Protecting Private Code in IDE Autocomplete using Differential Privacy

TL;DR

This paper explores using Differential Privacy to train code autocomplete models in IDEs, significantly enhancing privacy protections while maintaining high utility, thus enabling trustworthy AI-powered development tools.

Contribution

It demonstrates that Differential Privacy can effectively defend against membership inference attacks in code models with minimal utility loss.

Findings

DP reduces attack success rate close to random guessing

Model utility remains high with DP even on 100x less data

DP-trained model performs comparably to non-private models

Abstract

Modern Integrated Development Environments (IDEs) increasingly leverage Large Language Models (LLMs) to provide advanced features like code autocomplete. While powerful, training these models on user-written code introduces significant privacy risks, making the models themselves a new type of data vulnerability. Malicious actors can exploit this by launching attacks to reconstruct sensitive training data or infer whether a specific code snippet was used for training. This paper investigates the use of Differential Privacy (DP) as a robust defense mechanism for training an LLM for Kotlin code completion. We fine-tune a \texttt{Mellum} model using DP and conduct a comprehensive evaluation of its privacy and utility. Our results demonstrate that DP provides a strong defense against Membership Inference Attacks (MIAs), reducing the attack's success rate close to a random guess (AUC from 0.901 to 0.606). Furthermore, we show that this privacy guarantee comes at a minimal cost to model performance, with the DP-trained model achieving utility scores comparable to its non-private counterpart, even when trained on 100x less data. Our findings suggest that DP is a practical and effective solution for building private and trustworthy AI-powered IDE features.