Herculean: An Agentic Benchmark for Financial Intelligence

TL;DR

Herculean is a new comprehensive benchmark designed to evaluate AI agents' ability to perform complex, real-world financial workflows, highlighting current strengths and weaknesses in agentic financial intelligence.

Contribution

The paper introduces Herculean, the first benchmark for agentic financial intelligence covering four workflows with standardized environments for end-to-end assessment.

Findings

Agents perform well on Trading and Market Insights.

Agents struggle with Hedging and Auditing tasks.

Current agents lack reliable long-horizon coordination and structured verification.

Abstract

As AI agents improve, the central question is no longer whether they can solve isolated well-defined financial tasks, but whether they can reliably carry out financial professional work. Existing financial benchmarks offer only a partial view of this ability, as they primarily evaluate static competencies such as question answering, retrieval, summarization, and classification. We introduce Herculean, the first skilled benchmark for agentic financial intelligence spanning four representative workflows, including Trading, Hedging, Market Insights, and Auditing. Each workflow is instantiated as a standardized MCP-based skill environment with its own tools, interaction dynamics, constraints, and success criteria, enabling consistent end-to-end assessment of heterogeneous agent systems. Across frontier agents, we find agents perform relatively well on Trading and Market Insights, but struggle substantially on Hedging and Auditing, where long-horizon coordination, state consistency, and structured verification are critical. Overall, our results point to a key gap in current agents in turning financial reasoning into dependable workflow execution in high-stakes financial workflows.