AMO-Bench: Large Language Models Still Struggle in High School Math Competitions

TL;DR

AMO-Bench is a challenging new benchmark with Olympiad-level math problems designed to evaluate and push the limits of large language models' mathematical reasoning abilities, revealing significant performance gaps.

Contribution

This paper introduces AMO-Bench, a novel, expert-validated, high-difficulty math benchmark with original problems, specifically designed to assess top-tier LLMs beyond existing competition-based tests.

Findings

Top LLMs score only around 52.4% accuracy on AMO-Bench.

Most LLMs perform below 40%, indicating substantial room for improvement.

Performance improves with increased test-time compute, showing a promising scaling trend.

Abstract

We present AMO-Bench, an Advanced Mathematical reasoning benchmark with Olympiad level or even higher difficulty, comprising 50 human-crafted problems. Existing benchmarks have widely leveraged high school math competitions for evaluating mathematical reasoning capabilities of large language models (LLMs). However, many existing math competitions are becoming less effective for assessing top-tier LLMs due to performance saturation (e.g., AIME24/25). To address this, AMO-Bench introduces more rigorous challenges by ensuring all 50 problems are (1) cross-validated by experts to meet at least the International Mathematical Olympiad (IMO) difficulty standards, and (2) entirely original problems to prevent potential performance leakages from data memorization. Moreover, each problem in AMO-Bench requires only a final answer rather than a proof, enabling automatic and robust grading for evaluation. Experimental results across 26 LLMs on AMO-Bench show that even the best-performing model achieves only 52.4% accuracy on AMO-Bench, with most LLMs scoring below 40%. Beyond these poor performances, our further analysis reveals a promising scaling trend with increasing test-time compute on AMO-Bench. These results highlight the significant room for improving the mathematical reasoning in current LLMs. We release AMO-Bench to facilitate further research into advancing the reasoning abilities of language models. https://amo-bench.github.io/