A Systematic Study of Code Obfuscation Against LLM-based Vulnerability Detection

TL;DR

This paper systematically categorizes and evaluates various code obfuscation techniques across multiple programming languages and LLM models to understand their impact on vulnerability detection.

Contribution

It provides a comprehensive classification of obfuscation methods and a unified framework for assessing their effects on LLM-based vulnerability detection.

Findings

Obfuscation can both improve and degrade LLM detection performance.

Different obfuscation techniques have varied impacts depending on vulnerability and code properties.

The study highlights open challenges for making LLMs more robust against obfuscated code.

Abstract

As large language models (LLMs) are increasingly adopted for code vulnerability detection, their reliability and robustness across diverse vulnerability types have become a pressing concern. In traditional adversarial settings, code obfuscation has long been used as a general strategy to bypass auditing tools, preserving exploitability without tampering with the tools themselves. Numerous efforts have explored obfuscation methods and tools, yet their capabilities differ in terms of supported techniques, granularity, and programming languages, making it difficult to systematically assess their impact on LLM-based vulnerability detection. To address this gap, we provide a structured systematization of obfuscation techniques and evaluate them under a unified framework. Specifically, we categorize existing obfuscation methods into three major classes (layout, data flow, and control flow) covering 11 subcategories and 19 concrete techniques. We implement these techniques across four programming languages (Solidity, C, C++, and Python) using a consistent LLM-driven approach, and evaluate their effects on 15 LLMs spanning four model families (DeepSeek, OpenAI, Qwen, and LLaMA), as well as on two coding agents (GitHub Copilot and Codex). Our findings reveal both positive and negative impacts of code obfuscation on LLM-based vulnerability detection, highlighting conditions under which obfuscation leads to performance improvements or degradations. We further analyze these outcomes with respect to vulnerability characteristics, code properties, and model attributes. Finally, we outline several open problems and propose future directions to enhance the robustness of LLMs for real-world vulnerability detection.