Krikri: Advancing Open Large Language Models for Greek

TL;DR

Krikri introduces Llama-Krikri-8B, a large language model optimized for Greek, demonstrating superior performance in understanding and generating Greek language tasks through novel training and benchmarks.

Contribution

The paper presents a new Greek-specific LLM, Llama-Krikri-8B, with a multi-stage training pipeline and three novel benchmarks for Greek language evaluation.

Findings

Outperforms existing Greek and multilingual LLMs in understanding and generation.

Introduces three new benchmarks for Greek language tasks.

Demonstrates effectiveness in natural language understanding, generation, and code tasks.

Abstract

We introduce Llama-Krikri-8B, a cutting-edge Large Language Model tailored for the Greek language, built on Meta's Llama 3.1-8B. Llama-Krikri-8B has been extensively trained on high-quality Greek data to ensure superior adaptation to linguistic nuances. With 8 billion parameters, it offers advanced capabilities while maintaining efficient computational performance. Llama-Krikri-8B supports both Modern Greek and English, and is also equipped to handle polytonic text and Ancient Greek. The chat version of Llama-Krikri-8B features a multi-stage post-training pipeline, utilizing both human and synthetic instruction and preference data, by applying techniques such as MAGPIE. In addition, for evaluation, we propose three novel public benchmarks for Greek. Our evaluation on existing as well as the proposed benchmarks shows notable improvements over comparable Greek and multilingual LLMs in both natural language understanding and generation as well as code generation.