Casting a SPELL: Sentence Pairing Exploration for LLM Limitation-breaking

TL;DR

This paper introduces SPELL, a novel framework for systematically testing and revealing security weaknesses in LLMs' ability to generate malicious code, highlighting significant safety gaps.

Contribution

We propose SPELL, a comprehensive prompt construction framework that effectively uncovers security vulnerabilities in LLMs' malicious code generation capabilities.

Findings

SPELL achieves high attack success rates across multiple models.

Generated malicious code bypasses state-of-the-art detection systems.

Security gaps in current LLMs for code generation are significant.

Abstract

Large language models (LLMs) have revolutionized software development through AI-assisted coding tools, enabling developers with limited programming expertise to create sophisticated applications. However, this accessibility extends to malicious actors who may exploit these powerful tools to generate harmful software. Existing jailbreaking research primarily focuses on general attack scenarios against LLMs, with limited exploration of malicious code generation as a jailbreak target. To address this gap, we propose SPELL, a comprehensive testing framework specifically designed to evaluate the weakness of security alignment in malicious code generation. Our framework employs a time-division selection strategy that systematically constructs jailbreaking prompts by intelligently combining sentences from a prior knowledge dataset, balancing exploration of novel attack patterns with exploitation of successful techniques. Extensive evaluation across three advanced code models (GPT-4.1, Claude-3.5, and Qwen2.5-Coder) demonstrates SPELL's effectiveness, achieving attack success rates of 83.75%, 19.38%, and 68.12% respectively across eight malicious code categories. The generated prompts successfully produce malicious code in real-world AI development tools such as Cursor, with outputs confirmed as malicious by state-of-the-art detection systems at rates exceeding 73%. These findings reveal significant security gaps in current LLM implementations and provide valuable insights for improving AI safety alignment in code generation applications.