BERT Fine-tuning For Arabic Text Summarization

TL;DR

This paper demonstrates how to fine-tune multilingual BERT for Arabic text summarization, creating the first model for abstractive Arabic summarization and evaluating its performance in extractive tasks.

Contribution

It introduces the first Arabic abstractive summarization model using BERT fine-tuning and evaluates its effectiveness in Arabic extractive summarization.

Findings

Successful fine-tuning of multilingual BERT for Arabic summarization

First documented model for Arabic abstractive summarization

Performance evaluation on Arabic corpora

Abstract

Fine-tuning a pretrained BERT model is the state of the art method for extractive/abstractive text summarization, in this paper we showcase how this fine-tuning method can be applied to the Arabic language to both construct the first documented model for abstractive Arabic text summarization and show its performance in Arabic extractive summarization. Our model works with multilingual BERT (as Arabic language does not have a pretrained BERT of its own). We show its performance in English corpus first before applying it to Arabic corpora in both extractive and abstractive tasks.