HyperCLOVA X THINK Technical Report

TL;DR

HyperCLOVA X THINK is a large, reasoning-focused multilingual language model trained on extensive Korean and English data, achieving high performance on Korean benchmarks and surpassing GPT-4.1 in vision tasks, with efficient training and future open-source plans.

Contribution

Introduces HyperCLOVA X THINK, a novel reasoning-oriented large language model with advanced training techniques, multi-modal capabilities, and competitive benchmarks, while emphasizing efficiency and open-source readiness.

Findings

Achieves high scores on Korean benchmarks like KMMLU and KoBigBench.

Matches or exceeds GPT-4.1 on vision-augmented KCSAT STEM tasks.

Uses less training compute than comparable models.

Abstract

We introduce HyperCLOVA X THINK, the first reasoning-focused large language model in the HyperCLOVA X family, pre-trained on roughly $6$ trillion high-quality Korean, and English tokens, augmented with targeted synthetic Korean data. It was implemented as a compute-memory-balanced Peri-LN Transformer scaled with $\mu$P, pre-trained through a three-stage curriculum that expands the context window to $128$K tokens, and post-trained via supervised fine-tuning with Reinforcement Learning from Verifiable Rewards supports both detailed rationale and concise-answer modes. It delivers competitive performance against similarly sized models on Korea-focused benchmarks such as KMMLU, CSAT, KoBALT-700, HAERAE-1.0, and KoBigBench, while preserving robust bilingual consistency and translation quality. In addition, a vision-augmented variant matches or exceeds GPT-4.1 on the KCSAT STEM benchmark, all of which are achieved with substantially lower training compute than existing models of similar sizes. We also present a pruning and distillation technique that will soon be applied to HyperCLOVA X THINK for an open-source and business-friendly foundation model. Altogether, these capabilities position HyperCLOVA X THINK as a robust foundation for Korean AI innovation and a valuable resource for the global research community.