Magistral

TL;DR

Magistral introduces a new reasoning model and RL training pipeline for large language models, demonstrating that pure RL on text can enhance reasoning, instruction following, and multimodal understanding without relying on prior RL traces.

Contribution

The paper presents Magistral, a scalable RL pipeline built from scratch, and shows how pure RL on text data can improve LLM reasoning and multimodal capabilities.

Findings

RL on text data maintains or improves model capabilities

Pure RL training enhances reasoning and instruction following

Open-source Magistral Small for broader research use

Abstract

We introduce Magistral, Mistral's first reasoning model and our own scalable reinforcement learning (RL) pipeline. Instead of relying on existing implementations and RL traces distilled from prior models, we follow a ground up approach, relying solely on our own models and infrastructure. Notably, we demonstrate a stack that enabled us to explore the limits of pure RL training of LLMs, present a simple method to force the reasoning language of the model, and show that RL on text data alone maintains most of the initial checkpoint's capabilities. We find that RL on text maintains or improves multimodal understanding, instruction following and function calling. We present Magistral Medium, trained for reasoning on top of Mistral Medium 3 with RL alone, and we open-source Magistral Small (Apache 2.0) which further includes cold-start data from Magistral Medium.