# AI Reasoning Models for Problem Solving in Physics

## TL;DR

This paper evaluates the reasoning capabilities of a large language model, o3-mini, in solving introductory physics problems, demonstrating high success rates but with some difficulty on complex topics.

## Contribution

It provides an empirical assessment of a reasoning LLM's performance on physics problems, highlighting strengths and limitations across different topics.

## Key findings

- 94% problem-solving success rate
- Strong performance on mechanics topics
- Lower accuracy on waves and thermodynamics

## Abstract

Reasoning models are the new generation of Large Language Models (LLMs) capable of complex problem solving. Their reliability in solving introductory physics problems was tested by evaluating a sample of n = 5 solutions generated by one such model -- OpenAI's o3-mini -- per each problem from 20 chapters of a standard undergraduate textbook. In total, N = 408 problems were given to the model and N x n = 2,040 generated solutions examined. The model successfully solved 94% of the problems posed, excelling at the beginning topics in mechanics but struggling with the later ones such as waves and thermodynamics.