Security of Language Models for Code: A Systematic Literature Review

TL;DR

This paper systematically reviews 67 studies on the security vulnerabilities of language models for code, highlighting attack and defense strategies, datasets, tools, and future research directions.

Contribution

It provides the first comprehensive survey of security issues in CodeLMs, organizing existing research and identifying key challenges and open problems.

Findings

Attack strategies and defense mechanisms are evolving rapidly.

Common datasets and evaluation metrics are identified.

Open-source tools for security assessment are highlighted.

Abstract

Language models for code (CodeLMs) have emerged as powerful tools for code-related tasks, outperforming traditional methods and standard machine learning approaches. However, these models are susceptible to security vulnerabilities, drawing increasing research attention from domains such as software engineering, artificial intelligence, and cybersecurity. Despite the growing body of research focused on the security of CodeLMs, a comprehensive survey in this area remains absent. To address this gap, we systematically review 67 relevant papers, organizing them based on attack and defense strategies. Furthermore, we provide an overview of commonly used language models, datasets, and evaluation metrics, and highlight open-source tools and promising directions for future research in securing CodeLMs.