MedCalc-Eval and MedCalc-Env: Advancing Medical Calculation Capabilities of Large Language Models

TL;DR

This paper introduces MedCalc-Eval, a comprehensive benchmark for evaluating large language models' medical calculation skills, and MedCalc-Env, a reinforcement learning environment to enhance multi-step clinical reasoning, achieving state-of-the-art results.

Contribution

The paper presents the largest medical calculation benchmark and a novel RL environment for improving LLMs' quantitative reasoning in medicine.

Findings

Qwen2.5-32B fine-tuned in MedCalc-Env achieves state-of-the-art performance.

Benchmark covers diverse calculation tasks across multiple medical specialties.

Identifies remaining challenges like unit conversion and multi-condition reasoning.

Abstract

As large language models (LLMs) enter the medical domain, most benchmarks evaluate them on question answering or descriptive reasoning, overlooking quantitative reasoning critical to clinical decision-making. Existing datasets like MedCalc-Bench cover few calculation tasks and fail to reflect real-world computational scenarios. We introduce MedCalc-Eval, the largest benchmark for assessing LLMs' medical calculation abilities, comprising 700+ tasks across two types: equation-based (e.g., Cockcroft-Gault, BMI, BSA) and rule-based scoring systems (e.g., Apgar, Glasgow Coma Scale). These tasks span diverse specialties including internal medicine, surgery, pediatrics, and cardiology, offering a broader and more challenging evaluation setting. To improve performance, we further develop MedCalc-Env, a reinforcement learning environment built on the InternBootcamp framework, enabling multi-step clinical reasoning and planning. Fine-tuning a Qwen2.5-32B model within this environment achieves state-of-the-art results on MedCalc-Eval, with notable gains in numerical sensitivity, formula selection, and reasoning robustness. Remaining challenges include unit conversion, multi-condition logic, and contextual understanding. Code and datasets are available at https://github.com/maokangkun/MedCalc-Eval.