LLMs with in-context learning for Algorithmic Theoretical Physics

TL;DR

This paper explores using large language models combined with computer algebra systems to perform complex algorithmic tasks in theoretical physics, demonstrating promising capabilities and areas for improvement.

Contribution

It introduces a framework interfacing Claude with Maple for physics computations, showcasing the potential of LLMs with CAS in theoretical physics.

Findings

LLM with CAS can solve most test problems in cosmological perturbations

Worked examples significantly improve LLM performance

Current limitations and failure modes are identified and discussed

Abstract

There is an increasing number of algorithmic computations in theoretical physics. These, while conceptually simple, can nevertheless be time-consuming and contain subtleties that should not be overlooked. Given the recent improvement of Large Language Models (LLM), it is natural to investigate whether LLMs equipped with a computer algebra system (CAS) runtime and sufficiently informative context can reliably carry out these algorithmic tasks. In this work, we interface Claude with Maple, and apply this framework to cosmological perturbations in modified theories of gravity. We demonstrate the current capabilities of this approach, the typical failures, and how the same can be improved. We find that a frontier LLM supplied with worked examples is able to solve most test problems.