Time Travel is Cheating: Going Live with DeepFund for Real-Time Fund Investment Benchmarking

TL;DR

This paper introduces DeepFund, a real-time benchmarking tool for evaluating large language models in fund investment, revealing current LLM limitations in active trading through live market testing.

Contribution

We develop DeepFund, a live, leakage-free benchmark connecting LLMs with real-time market data, enabling accurate assessment of their practical fund management capabilities.

Findings

LLMs like DeepSeek-V3 and Claude-3.7-Sonnet incur net trading losses in real-time evaluation.

DeepFund exposes significant practical challenges for LLMs in active fund management.

Current LLMs have limited effectiveness in real-time trading scenarios.

Abstract

Large Language Models (LLMs) have demonstrated notable capabilities across financial tasks, including financial report summarization, earnings call transcript analysis, and asset classification. However, their real-world effectiveness in managing complex fund investment remains inadequately assessed. A fundamental limitation of existing benchmarks for evaluating LLM-driven trading strategies is their reliance on historical back-testing, inadvertently enabling LLMs to "time travel"-leveraging future information embedded in their training corpora, thus resulting in possible information leakage and overly optimistic performance estimates. To address this issue, we introduce DeepFund, a live fund benchmark tool designed to rigorously evaluate LLM in real-time market conditions. Utilizing a multi-agent architecture, DeepFund connects directly with real-time stock market data-specifically data published after each model pretraining cutoff-to ensure fair and leakage-free evaluations. Empirical tests on nine flagship LLMs from leading global institutions across multiple investment dimensions-including ticker-level analysis, investment decision-making, portfolio management, and risk control-reveal significant practical challenges. Notably, even cutting-edge models such as DeepSeek-V3 and Claude-3.7-Sonnet incur net trading losses within DeepFund real-time evaluation environment, underscoring the present limitations of LLMs for active fund management. Our code is available at https://github.com/HKUSTDial/DeepFund.