TookaBERT: A Step Forward for Persian NLU

TL;DR

TookaBERT introduces two new Persian BERT models trained on Persian data, outperforming existing models on 14 NLU tasks with significant improvements, advancing Persian natural language understanding.

Contribution

The paper presents two novel Persian BERT models trained on Persian data, demonstrating superior performance over seven existing models across multiple NLU tasks.

Findings

Our larger model outperforms competitors by at least +2.8 points on average.

The models show strong effectiveness across diverse Persian NLU tasks.

TookaBERT advances Persian NLP with improved model performance.

Abstract

The field of natural language processing (NLP) has seen remarkable advancements, thanks to the power of deep learning and foundation models. Language models, and specifically BERT, have been key players in this progress. In this study, we trained and introduced two new BERT models using Persian data. We put our models to the test, comparing them to seven existing models across 14 diverse Persian natural language understanding (NLU) tasks. The results speak for themselves: our larger model outperforms the competition, showing an average improvement of at least +2.8 points. This highlights the effectiveness and potential of our new BERT models for Persian NLU tasks.