AlignTune: Modular Toolkit for Post-Training Alignment of Large Language Models

TL;DR

AlignTune is a modular toolkit that streamlines post-training alignment of large language models by standardizing workflows, enabling reproducible experiments, and supporting flexible optimization methods.

Contribution

It introduces a unified, extensible toolkit for LLM alignment that addresses backend interference and reproducibility issues in current practices.

Findings

Standardizes alignment workflows across different backends

Enables controlled, reproducible experiments in LLM alignment

Supports both supervised fine-tuning and RLHF-style optimization

Abstract

Post-training alignment is central to deploying large language models (LLMs), yet practical workflows remain split across backend-specific tools and ad-hoc glue code, making experiments hard to reproduce. We identify backend interference, reward fragmentation, and irreproducible pipelines as key obstacles in alignment research. We introduce AlignTune, a modular toolkit exposing a unified interface for supervised fine-tuning (SFT) and RLHF-style optimization with interchangeable TRL and Unsloth backends. AlignTune standardizes configuration, provides an extensible reward layer (rule-based and learned), and integrates evaluation over standard benchmarks and custom tasks. By isolating backend-specific logic behind a single factory boundary, AlignTune enables controlled comparisons and reproducible alignment experiments.