Jailbreak-as-a-Service++: Unveiling Distributed AI-Driven Malicious Information Campaigns Powered by LLM Crowdsourcing

TL;DR

This paper uncovers how malicious actors can exploit distributed LLMs via crowdsourcing to generate harmful content, demonstrating the challenges in regulating such distributed AI-driven malicious campaigns.

Contribution

It introduces PoisonSwarm, a novel framework showing how attackers can coordinate LLMs to produce malicious content, highlighting vulnerabilities in MaaS ecosystems.

Findings

PoisonSwarm outperforms existing methods in data quality and diversity.

Distributed misuse is difficult to regulate, requiring ecosystem-level defenses.

Experiments confirm the effectiveness of the proposed approach.

Abstract

To prevent the misuse of Large Language Models (LLMs) for malicious purposes, numerous efforts have been made to develop the safety alignment mechanisms of LLMs. However, as multiple LLMs become readily accessible through various Model-as-a-Service (MaaS) platforms, attackers can strategically exploit LLMs' heterogeneous safety policies to fulfill malicious information generation tasks in a distributed manner. In this study, we introduce \textit{\textbf{PoisonSwarm}} to how attackers can reliably launder malicious tasks via the speculative use of LLM crowdsourcing. Building upon a scheduler orchestrating crowdsourced LLMs, PoisonSwarm maps the given malicious task to a benign analogue to derive a content template, decomposes it into semantic units for crowdsourced unit-wise rewriting, and reassembles the outputs into malicious content. Experiments show its superiority over existing methods in data quality, diversity, and success rates. Regulation simulations further reveal the difficulty of governing such distributed, orchestrated misuse in MaaS ecosystems, highlighting the need for coordinated, ecosystem-level defenses.