RoLargeSum: A Large Dialect-Aware Romanian News Dataset for Summary, Headline, and Keyword Generation

TL;DR

This paper introduces RoLargeSum, a large-scale Romanian news dataset with summaries, headlines, and keywords, enabling better development of summarization models for Romanian and similar languages.

Contribution

The creation of RoLargeSum, a comprehensive Romanian news dataset with rich metadata, and benchmarking of various language models on this dataset.

Findings

BART variants achieved promising results on Romanian summarization.

Open-source large language models showed potential but need further refinement.

Manual evaluation revealed specific challenges in Romanian summarization quality.

Abstract

Using supervised automatic summarisation methods requires sufficient corpora that include pairs of documents and their summaries. Similarly to many tasks in natural language processing, most of the datasets available for summarization are in English, posing challenges for developing summarization models in other languages. Thus, in this work, we introduce RoLargeSum, a novel large-scale summarization dataset for the Romanian language crawled from various publicly available news websites from Romania and the Republic of Moldova that were thoroughly cleaned to ensure a high-quality standard. RoLargeSum contains more than 615K news articles, together with their summaries, as well as their headlines, keywords, dialect, and other metadata that we found on the targeted websites. We further evaluated the performance of several BART variants and open-source large language models on RoLargeSum for benchmarking purposes. We manually evaluated the results of the best-performing system to gain insight into the potential pitfalls of this data set and future development.