CodeIP: A Grammar-Guided Multi-Bit Watermark for Large Language Models of Code

TL;DR

CodeIP introduces a multi-bit watermarking method for large language models generating code, enabling IP protection and model identification without compromising code syntax, demonstrated across multiple programming languages.

Contribution

The paper presents a novel multi-bit watermarking technique for code generated by LLMs, including a type predictor to ensure syntactical correctness, advancing beyond single-bit watermarking methods.

Findings

Effective multi-bit watermarking across five programming languages

Maintains syntactical correctness of generated code

Successfully identifies LLM vendor ID in watermarked code

Abstract

Large Language Models (LLMs) have achieved remarkable progress in code generation. It now becomes crucial to identify whether the code is AI-generated and to determine the specific model used, particularly for purposes such as protecting Intellectual Property (IP) in industry and preventing cheating in programming exercises. To this end, several attempts have been made to insert watermarks into machine-generated code. However, existing approaches are limited to inserting only a single bit of information. In this paper, we introduce CodeIP, a novel multi-bit watermarking technique that inserts additional information to preserve crucial provenance details, such as the vendor ID of an LLM, thereby safeguarding the IPs of LLMs in code generation. Furthermore, to ensure the syntactical correctness of the generated code, we propose constraining the sampling process for predicting the next token by training a type predictor. Experiments conducted on a real-world dataset across five programming languages demonstrate the effectiveness of CodeIP in watermarking LLMs for code generation while maintaining the syntactical correctness of code.