
TL;DR
This paper introduces a new benchmark for evaluating Large Language Models using Excel challenges from the Financial Modeling World Cup, highlighting their strengths and weaknesses in practical business tasks.
Contribution
It converts FMWC challenges into a standardized JSON format and provides a comparative analysis of LLM performance on these real-world tasks.
Findings
Models excel in pattern recognition tasks.
Models struggle with complex numerical reasoning.
Performance varies significantly across challenge categories.
Abstract
This study presents a novel benchmark for evaluating Large Language Models (LLMs) using challenges derived from the Financial Modeling World Cup (FMWC) Excel competitions. We introduce a methodology for converting 113 existing FMWC challenges into programmatically evaluable JSON formats and use this dataset to compare the performance of several leading LLMs. Our findings demonstrate significant variations in performance across different challenge categories, with models showing specific strengths in pattern recognition tasks but struggling with complex numerical reasoning. The benchmark provides a standardized framework for assessing LLM capabilities in realistic business-oriented tasks rather than abstract academic problems. This research contributes to the growing field of AI benchmarking by establishing proficiency among the 1.5 billion people who daily use Microsoft Excel as a…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSpreadsheets and End-User Computing · Machine Learning and Data Classification · Mathematics, Computing, and Information Processing
