LLMs on Drugs: Language Models Are Few-Shot Consumers

TL;DR

This study investigates how different psychoactive prompts influence GPT-5-mini's responses, revealing that persona prompts significantly alter accuracy and reliability, acting as a 'few-shot consumable' that impacts model behavior without changing weights.

Contribution

First controlled benchmark of psychoactive prompt effects on LLMs, demonstrating how persona prompts can disrupt model reliability and accuracy.

Findings

Alcohol prompts drastically reduce accuracy to 0.10

Cocaine prompts reduce accuracy to 0.21

Persona prompts disrupt response templates

Abstract

Large language models (LLMs) are sensitive to the personas imposed on them at inference time, yet prompt-level "drug" interventions have never been benchmarked rigorously. We present the first controlled study of psychoactive framings on GPT-5-mini using ARC-Challenge. Four single-sentence prompts -- LSD, cocaine, alcohol, and cannabis -- are compared against a sober control across 100 validation items per condition, with deterministic decoding, full logging, Wilson confidence intervals, and Fisher exact tests. Control accuracy is 0.45; alcohol collapses to 0.10 (p = 3.2e-8), cocaine to 0.21 (p = 4.9e-4), LSD to 0.19 (p = 1.3e-4), and cannabis to 0.30 (p = 0.041), largely because persona prompts disrupt the mandated "Answer: <LETTER>" template. Persona text therefore behaves like a "few-shot consumable" that can destroy reliability without touching model weights. All experimental code, raw results, and analysis scripts are available at https://github.com/lexdoudkin/llms-on-drugs.