Can A Gamer Train A Mathematical Reasoning Model?

TL;DR

This paper shows that a single gaming GPU can effectively train a 1.5B parameter mathematical reasoning model, challenging the need for large-scale infrastructure and making high-performance AI research more accessible.

Contribution

The authors demonstrate training a competitive mathematical reasoning model using only a gaming GPU, integrating reinforcement learning and memory optimization techniques.

Findings

Achieved comparable or better performance on benchmarks with a 1.5B model.

Successfully trained on RTX 3080 Ti with 16GB memory.

Challenged the assumption that large infrastructure is necessary for high-quality mathematical reasoning models.

Abstract

While large language models (LLMs) have achieved remarkable performance in various tasks including mathematical reasoning, their development typically demands prohibitive computational resources. Recent advancements have reduced costs for training capable models, yet even these approaches rely on high-end hardware clusters. In this paper, we demonstrate that a single average gaming GPU can train a solid mathematical reasoning model, by integrating reinforcement learning and memory optimization techniques. Specifically, we train a 1.5B parameter mathematical reasoning model on RTX 3080 Ti of 16GB memory that achieves comparable or better performance on mathematical reasoning benchmarks than models several times larger, in resource-constrained environments. Our results challenge the paradigm that state-of-the-art mathematical reasoning necessitates massive infrastructure, democratizing access to high-performance AI research. https://github.com/shinandrew/YouronMath.