Profit is the Red Team: Stress-Testing Agents in Strategic Economic Interactions

TL;DR

This paper introduces a profit-driven red teaming method that trains adaptive adversaries to stress-test economic agents, revealing vulnerabilities and improving robustness without relying on predefined attack labels.

Contribution

It presents a novel, label-free red teaming protocol using learned opponents to identify exploit strategies in structured economic interactions.

Findings

Agents become more exploitable under profit-optimized red teaming.

Learned opponents discover probing, anchoring, and deception tactics.

Distilling exploit episodes into prompt rules enhances agent robustness.

Abstract

As agentic systems move into real-world deployments, their decisions increasingly depend on external inputs such as retrieved content, tool outputs, and information provided by other actors. When these inputs can be strategically shaped by adversaries, the relevant security risk extends beyond a fixed library of prompt attacks to adaptive strategies that steer agents toward unfavorable outcomes. We propose profit-driven red teaming, a stress-testing protocol that replaces handcrafted attacks with a learned opponent trained to maximize its profit using only scalar outcome feedback. The protocol requires no LLM-as-judge scoring, attack labels, or attack taxonomy, and is designed for structured settings with auditable outcomes. We instantiate it in a lean arena of four canonical economic interactions, which provide a controlled testbed for adaptive exploitability. In controlled experiments, agents that appear strong against static baselines become consistently exploitable under profit-optimized pressure, and the learned opponent discovers probing, anchoring, and deceptive commitments without explicit instruction. We then distill exploit episodes into concise prompt rules for the agent, which make most previously observed failures ineffective and substantially improve target performance. These results suggest that profit-driven red-team data can provide a practical route to improving robustness in structured agent settings with auditable outcomes.