OpenLLM-Ro -- Technical Report on Open-source Romanian LLMs

TL;DR

This paper introduces OpenLLM-Ro, the first Romanian-focused foundational and chat LLM, addressing the language performance gap by training and evaluating models specifically for Romanian language tasks.

Contribution

It presents the development and evaluation of the first Romanian-specific foundational and chat LLM, filling a gap in multilingual LLM research.

Findings

Achieved competitive performance on Romanian NLP tasks

Demonstrated effectiveness of language-specific training

Provided benchmarks for Romanian LLMs

Abstract

In recent years, Large Language Models (LLMs) have achieved almost human-like performance on various tasks. While some LLMs have been trained on multilingual data, most of the training data is in English. Hence, their performance in English greatly exceeds their performance in other languages. This document presents our approach to training and evaluating the first foundational and chat LLM specialized for Romanian.