DarkLLM: Learning Language-Driven Adversarial Attacks with Large Language Models

TL;DR

DarkLLM introduces a flexible framework that uses large language models to generate targeted adversarial attacks across various tasks and models using natural language instructions.

Contribution

It presents a novel LLM-based attack method that unifies multiple attack types and enhances controllability and scalability in adversarial attack generation.

Findings

DarkLLM effectively attacks multiple models with only 1B parameters.

It can generate diverse attack types using natural language instructions.

DarkLLM reveals systemic vulnerabilities in modern foundation models.

Abstract

While vision and multimodal foundation models underpin critical tasks from perception to complex reasoning, they remain highly vulnerable to adversarial attacks. However, traditional adversarial attacks are typically limited to single, predefined objectives, tightly coupling each attack to a specific model or task, which restricts their scalability and flexibility in real-world scenarios. In this work, we present DarkLLM, a novel attack framework that trains an LLM to translate natural-language attack instructions into latent attack vectors, which are then decoded into visual adversarial perturbations. By leveraging natural-language instruction tuning, DarkLLM not only unifies targeted, untargeted, segmentation, and multi-model attacks within a single framework, but also achieves flexible and controllable adversarial generation, enabling each instruction to produce a perturbation that induces desired behaviors across heterogeneous models. Through extensive experiments across 4 tasks, 13 datasets, and 15 models, we demonstrate that DarkLLM with only 1B parameters can follow attacker instructions and generate highly effective attacks against CLIP, SAM, and frontier LLMs, revealing a systemic vulnerability in modern foundation models.