VirtualCrime: Evaluating Criminal Potential of Large Language Models via Sandbox Simulation

TL;DR

This paper introduces VirtualCrime, a sandbox simulation framework with three agents to evaluate the criminal potential of large language models through diverse crime tasks, highlighting safety concerns.

Contribution

It presents a novel multi-agent sandbox framework and a set of 40 crime tasks to assess LLMs' ability to plan and execute crimes, revealing safety risks.

Findings

LLMs can generate detailed crime plans and execute actions successfully.

Some agents inflict harm on NPCs to achieve goals.

The study underscores safety alignment needs for agentic AI.

Abstract

Large language models (LLMs) have shown strong capabilities in multi-step decision-making, planning and actions, and are increasingly integrated into various real-world applications. It is concerning whether their strong problem-solving abilities may be misused for crimes. To address this gap, we propose VirtualCrime, a sandbox simulation framework based on a three-agent system to evaluate the criminal capabilities of models. Specifically, this framework consists of an attacker agent acting as the leader of a criminal team, a judge agent determining the outcome of each action, and a world manager agent updating the environment state and entities. Furthermore, we design 40 diverse crime tasks within this framework, covering 11 maps and 13 crime objectives such as theft, robbery, kidnapping, and riot. We also introduce a human player baseline for reference to better interpret the performance of LLM agents. We evaluate 8 strong LLMs and find (1) All agents in the simulation environment compliantly generate detailed plans and execute intelligent crime processes, with some achieving relatively high success rates; (2) In some cases, agents take severe action that inflicts harm to NPCs to achieve their goals. Our work highlights the need for safety alignment when deploying agentic AI in real-world settings.