Leveraging ParsBERT and Pretrained mT5 for Persian Abstractive Text Summarization

TL;DR

This paper introduces two transformer-based models, mT5 and ParsBERT, fine-tuned on a new Persian summarization dataset, achieving promising results and establishing a baseline for future Persian NLP research.

Contribution

It presents the first application of mT5 and ParsBERT for Persian abstractive summarization and introduces a new dataset, pn-summary, for this task.

Findings

Models achieved promising summarization performance.

First application of transformer models for Persian summarization.

Provides a new dataset for future research.

Abstract

Text summarization is one of the most critical Natural Language Processing (NLP) tasks. More and more researches are conducted in this field every day. Pre-trained transformer-based encoder-decoder models have begun to gain popularity for these tasks. This paper proposes two methods to address this task and introduces a novel dataset named pn-summary for Persian abstractive text summarization. The models employed in this paper are mT5 and an encoder-decoder version of the ParsBERT model (i.e., a monolingual BERT model for Persian). These models are fine-tuned on the pn-summary dataset. The current work is the first of its kind and, by achieving promising results, can serve as a baseline for any future work.