Can LLM-based Financial Investing Strategies Outperform the Market in Long Run?

TL;DR

This study critically evaluates LLM-based financial investing strategies over long periods and broad stock universes, revealing their limited robustness and the need for improved regime-aware risk management.

Contribution

It introduces FINSABER, a comprehensive backtesting framework, and provides a rigorous assessment of LLM strategies' performance across extended timeframes and diverse stocks.

Findings

LLM strategies' advantages diminish over longer periods and broader universes.

They tend to be overly conservative in bull markets and overly aggressive in bear markets.

Performance deteriorates significantly when evaluated beyond narrow, short-term datasets.

Abstract

Large Language Models (LLMs) have recently been leveraged for asset pricing tasks and stock trading applications, enabling AI agents to generate investment decisions from unstructured financial data. However, most evaluations of LLM timing-based investing strategies are conducted on narrow timeframes and limited stock universes, overstating effectiveness due to survivorship and data-snooping biases. We critically assess their generalizability and robustness by proposing FINSABER, a backtesting framework evaluating timing-based strategies across longer periods and a larger universe of symbols. Systematic backtests over two decades and 100+ symbols reveal that previously reported LLM advantages deteriorate significantly under broader cross-section and over a longer-term evaluation. Our market regime analysis further demonstrates that LLM strategies are overly conservative in bull markets, underperforming passive benchmarks, and overly aggressive in bear markets, incurring heavy losses. These findings highlight the need to develop LLM strategies that are able to prioritise trend detection and regime-aware risk controls over mere scaling of framework complexity.