torchtune: PyTorch native post-training library

TL;DR

torchtune is a PyTorch-native library that simplifies and enhances the post-training process of large language models, focusing on modularity, transparency, and efficiency.

Contribution

It introduces a flexible, transparent, and efficient post-training library for LLMs that outperforms existing frameworks in performance and memory usage.

Findings

Provides strong performance across various settings

Achieves better memory efficiency than competitors

Maintains high flexibility for research iteration

Abstract

Modern LLMs typically require multistage training pipelines to achieve strong downstream performance, with post-training serving as the main interface for adapting open-weight models. We introduce torchtune, a PyTorch-native library designed to streamline the post-training lifecycle of LLMs, enabling efficient fine-tuning, experimentation, and deployment-oriented workflows. Unlike many existing fine-tuning frameworks, which often optimize for ease of use, specialized recipes, or hardware efficiency at the cost of transparency and extensibility, torchtune emphasizes modularity, hackability, and direct access to the underlying PyTorch components. In this paper, we present the design principles behind torchtune, describe how they are reflected in its model builders, training recipes, and distributed training stack, and evaluate the library across representative post-training settings. We compare against popular fine-tuning frameworks, including Axolotl and Unsloth, and show that torchtune provides strong performance and memory efficiency across many settings while remaining flexible enough for rapid research iteration. These results position torchtune as a practical foundation for reproducible LLMs post-training research.