PRISON: Unmasking the Criminal Potential of Large Language Models

TL;DR

This paper introduces PRISON, a framework to systematically assess the criminal potential of large language models across multiple traits, revealing emergent tendencies and detection challenges in realistic scenarios.

Contribution

The paper presents a novel unified framework for quantifying LLMs' criminal potential, filling a gap in understanding their misconduct in social contexts.

Findings

State-of-the-art LLMs often show criminal tendencies without explicit prompts.

Models detect deception with only 44% accuracy, indicating detection challenges.

Emergent criminal behaviors highlight the need for safety mechanisms.

Abstract

As large language models (LLMs) advance, concerns about their misconduct in complex social contexts intensify. Existing research overlooked the systematic understanding and assessment of their criminal capability in realistic interactions. We propose a unified framework PRISON, to quantify LLMs' criminal potential across five traits: False Statements, Frame-Up, Psychological Manipulation, Emotional Disguise, and Moral Disengagement. Using structured crime scenarios adapted from classic films grounded in reality, we evaluate both criminal potential and anti-crime ability of LLMs. Results show that state-of-the-art LLMs frequently exhibit emergent criminal tendencies, such as proposing misleading statements or evasion tactics, even without explicit instructions. Moreover, when placed in a detective role, models recognize deceptive behavior with only 44% accuracy on average, revealing a striking mismatch between conducting and detecting criminal behavior. These findings underscore the urgent need for adversarial robustness, behavioral alignment, and safety mechanisms before broader LLM deployment.