State-of-the-art Small Language Coder Model: Mify-Coder

TL;DR

Mify-Coder is a compact 2.5B-parameter code model trained on a large dataset, achieving near state-of-the-art performance in code generation and safety, rivaling larger models through efficient training and data strategies.

Contribution

The paper introduces Mify-Coder, a small yet high-performing code model trained with a compute-optimal approach, demonstrating that smaller models can match larger ones in coding tasks.

Findings

Mify-Coder achieves comparable accuracy to larger models.

Quantized variants enable deployment on standard desktops.

Efficient training pipeline enhances data quality and model performance.

Abstract

We present Mify-Coder, a 2.5B-parameter code model trained on 4.2T tokens using a compute-optimal strategy built on the Mify-2.5B foundation model. Mify-Coder achieves comparable accuracy and safety while significantly outperforming much larger baseline models on standard coding and function-calling benchmarks, demonstrating that compact models can match frontier-grade models in code generation and agent-driven workflows. Our training pipeline combines high-quality curated sources with synthetic data generated through agentically designed prompts, refined iteratively using enterprise-grade evaluation datasets. LLM-based quality filtering further enhances data density, enabling frugal yet effective training. Through disciplined exploration of CPT-SFT objectives, data mixtures, and sampling dynamics, we deliver frontier-grade code intelligence within a single continuous training trajectory. Empirical evidence shows that principled data and compute discipline allow smaller models to achieve competitive accuracy, efficiency, and safety compliance. Quantized variants of Mify-Coder enable deployment on standard desktop environments without requiring specialized hardware.