Can LLM Agents Be CFOs? Benchmarking Long-Horizon Resource Allocation in an Uncertain Enterprise Environment

TL;DR

This paper evaluates the ability of large language model (LLM) agents to perform long-term resource allocation in uncertain enterprise environments using a new CFO simulator, revealing significant robustness gaps.

Contribution

Introduces EnterpriseArena, a comprehensive CFO simulation environment for benchmarking LLM agents' long-horizon resource management under uncertainty.

Findings

Only 15.4% of trials survive the full horizon.

Larger models do not consistently outperform smaller ones.

Failures cascade across observation, timing, and capital sizing.

Abstract

Large language model (LLM) agents are increasingly tested on complex tasks, but their ability to allocate scarce resources over long horizons remains unclear. Unlike reactive tasks with immediate feedback, this setting requires agents to make binding commitments under partial observability, delayed consequences, hard resource budgets, and shifting dynamics. We introduce EnterpriseArena, a 132-month CFO simulator that evaluates long-horizon resource allocation under uncertainty in a FinTech lending firm. Agents must manage liquidity, close books, gather costly signals, and request equity or debt financing across changing macroeconomic regimes. The simulator is built from transformed firm-level financial data, anonymized business documents, decade-scale macroeconomic and industry signals, and expert-validated operating rules. Experiments across 23 LLMs and four agent frameworks show that current agents remain far from robust: only 15.4% of trials survive the full horizon, larger models do not reliably outperform smaller ones, and failures cascade across observation, action timing, and capital sizing. These findings establish long-horizon resource allocation under uncertainty as a distinct capability gap for LLM agents.