EngGPT2: Sovereign, Efficient and Open Intelligence

TL;DR

EngGPT2-16B-A3B is a resource-efficient, open, and European-focused Mixture-of-Experts language model trained on 2.5 trillion tokens, achieving competitive performance with reduced training and inference costs.

Contribution

This paper introduces EngGPT2, a novel 16-billion-parameter MoE model optimized for efficiency, multilingual reasoning, and European NLP tasks, with a focus on open and sovereign AI development.

Findings

Achieves performance comparable to larger models on key benchmarks.

Requires significantly less training data and inference power.

Supports multiple reasoning modes, including real-time turbo-reasoning.

Abstract

EngGPT2-16B-A3B is the latest iteration of Engineering Group's Italian LLM and it's built to be a Sovereign, Efficient and Open model. EngGPT2 is trained on 2.5 trillion tokens - less than Qwen3's 36T or Llama3's 15T - and delivers performance on key benchmarks, including MMLU-Pro, GSM8K, IFEval and HumanEval, comparable to dense models in the 8B-16B range, while requiring one-fifth to half of the inference power, and between one-tenth to one-sixth of the training data and consequent needed training power. Designed as a trained-from-scratch Mixture-of-Experts (MoE) architecture, EngGPT2 features 16 billion parameters with 3 billion active per inference, with expert sizes positioned between those used in GPT-OSS and Qwen3. Approximately 25% of its training corpus consists of Italian-language data, to deliver strong capabilities for European and Italian NLP tasks among models of similar scale. This efficiency aims to position EngGPT2 as a key contributor to the growing portfolio of open-weight European models, combining performance and efficiency with full alignment to the EU AI Act. EngGPT2 is also a single model capable of multiple reasoning modes: non-reasoning, reasoning in Italian or English, and turbo-reasoning (a concise, bullet-point style reasoning available in both languages designed for real-time reasoning use cases). EngGPT2 aims to set a new standard for resource-conscious, high-performance LLMs tailored to European and Italian contexts.