INTEGRALBENCH: Benchmarking LLMs with Definite Integral Problems

TL;DR

INTEGRALBENCH is a specialized benchmark for assessing large language models' ability to solve definite integral problems, providing symbolic and numerical solutions along with difficulty annotations to measure performance gaps.

Contribution

It introduces a novel benchmark with annotated difficulty levels for evaluating LLMs on definite integrals, advancing automated mathematical reasoning.

Findings

Significant performance gaps among state-of-the-art LLMs.

Strong correlation between problem difficulty and model accuracy.

Established baseline metrics for integral problem-solving by LLMs.

Abstract

We present INTEGRALBENCH, a focused benchmark designed to evaluate Large Language Model (LLM) performance on definite integral problems. INTEGRALBENCH provides both symbolic and numerical ground truth solutions with manual difficulty annotations. Our evaluation of nine state-of-the-art LLMs reveals significant performance gaps and strong correlations between problem difficulty and model accuracy, establishing baseline metrics for this challenging domain. INTEGRALBENCH aims to advance automated mathematical reasoning by providing a rigorous evaluation framework specifically tailored for definite integral computation.