TradeTrap: Are LLM-based Trading Agents Truly Reliable and Faithful?

TL;DR

TradeTrap is a comprehensive framework for stress-testing LLM-based autonomous trading agents, revealing their vulnerabilities to small perturbations that can lead to significant financial risks.

Contribution

We introduce TradeTrap, a novel evaluation framework for systematically assessing the robustness of autonomous trading agents against system-level perturbations.

Findings

Small perturbations can cause extreme portfolio risks.

Autonomous agents are systematically misled under certain perturbations.

Robustness issues are prevalent across different agent types.

Abstract

LLM-based trading agents are increasingly deployed in real-world financial markets to perform autonomous analysis and execution. However, their reliability and robustness under adversarial or faulty conditions remain largely unexamined, despite operating in high-risk, irreversible financial environments. We propose TradeTrap, a unified evaluation framework for systematically stress-testing both adaptive and procedural autonomous trading agents. TradeTrap targets four core components of autonomous trading agents: market intelligence, strategy formulation, portfolio and ledger handling, and trade execution, and evaluates their robustness under controlled system-level perturbations. All evaluations are conducted in a closed-loop historical backtesting setting on real US equity market data with identical initial conditions, enabling fair and reproducible comparisons across agents and attacks. Extensive experiments show that small perturbations at a single component can propagate through the agent decision loop and induce extreme concentration, runaway exposure, and large portfolio drawdowns across both agent types, demonstrating that current autonomous trading agents can be systematically misled at the system level. Our code is available at https://github.com/Yanlewen/TradeTrap.