CodeMark: Imperceptible Watermarking for Code Datasets against Neural Code Completion Models

TL;DR

CodeMark introduces an imperceptible watermarking technique for code datasets, enabling copyright protection and usage tracing in neural code completion models without affecting code functionality or model performance.

Contribution

The paper presents a novel semantic-preserving watermarking method specifically designed for code datasets, addressing a gap in existing watermarking techniques for natural language and images.

Findings

CodeMark maintains code functionality and model accuracy.

Watermarks are robust against attempts to remove or alter them.

The method is effective and imperceptible in practical evaluations.

Abstract

Code datasets are of immense value for training neural-network-based code completion models, where companies or organizations have made substantial investments to establish and process these datasets. Unluckily, these datasets, either built for proprietary or public usage, face the high risk of unauthorized exploits, resulting from data leakages, license violations, etc. Even worse, the ``black-box'' nature of neural models sets a high barrier for externals to audit their training datasets, which further connives these unauthorized usages. Currently, watermarking methods have been proposed to prohibit inappropriate usage of image and natural language datasets. However, due to domain specificity, they are not directly applicable to code datasets, leaving the copyright protection of this emerging and important field of code data still exposed to threats. To fill this gap, we propose a method, named CodeMark, to embed user-defined imperceptible watermarks into code datasets to trace their usage in training neural code completion models. CodeMark is based on adaptive semantic-preserving transformations, which preserve the exact functionality of the code data and keep the changes covert against rule-breakers. We implement CodeMark in a toolkit and conduct an extensive evaluation of code completion models. CodeMark is validated to fulfill all desired properties of practical watermarks, including harmlessness to model accuracy, verifiability, robustness, and imperceptibility.