Advanced Financial Reasoning at Scale: A Comprehensive Evaluation of Large Language Models on CFA Level III

TL;DR

This paper evaluates 23 advanced large language models on the CFA Level III exam, demonstrating significant progress in financial reasoning capabilities and providing insights for responsible deployment in financial sectors.

Contribution

It offers a comprehensive benchmark of LLMs on CFA Level III, introducing a stricter grading methodology and analyzing model performance for high-stakes financial reasoning.

Findings

Leading models scored over 77% on CFA Level III.

Progress shown in LLMs' ability for complex financial reasoning.

Highlights challenges in deployment and nuanced interpretation.

Abstract

As financial institutions increasingly adopt Large Language Models (LLMs), rigorous domain-specific evaluation becomes critical for responsible deployment. This paper presents a comprehensive benchmark evaluating 23 state-of-the-art LLMs on the Chartered Financial Analyst (CFA) Level III exam - the gold standard for advanced financial reasoning. We assess both multiple-choice questions (MCQs) and essay-style responses using multiple prompting strategies including Chain-of-Thought and Self-Discover. Our evaluation reveals that leading models demonstrate strong capabilities, with composite scores such as 79.1% (o4-mini) and 77.3% (Gemini 2.5 Flash) on CFA Level III. These results, achieved under a revised, stricter essay grading methodology, indicate significant progress in LLM capabilities for high-stakes financial applications. Our findings provide crucial guidance for practitioners on model selection and highlight remaining challenges in cost-effective deployment and the need for nuanced interpretation of performance against professional benchmarks.