RigoBERTa: A State-of-the-Art Language Model For Spanish

TL;DR

RigoBERTa is a new Spanish language model based on DeBERTa architecture that outperforms existing models on most natural language understanding tasks, setting new state-of-the-art results.

Contribution

The paper introduces RigoBERTa, a novel Spanish language model utilizing DeBERTa architecture, trained on a curated corpus, and demonstrates its superior performance over existing models.

Findings

RigoBERTa outperforms MarIA, BERTIN, and BETO on 10 of 13 NLU tasks.

Achieves new state-of-the-art results in Spanish language understanding.

Demonstrates advantages of DeBERTa architecture for Spanish NLP.

Abstract

This paper presents RigoBERTa, a State-of-the-Art Language Model for Spanish. RigoBERTa is trained over a well-curated corpus formed up from different subcorpora with key features. It follows the DeBERTa architecture, which has several advantages over other architectures of similar size as BERT or RoBERTa. RigoBERTa performance is assessed over 13 NLU tasks in comparison with other available Spanish language models, namely, MarIA, BERTIN and BETO. RigoBERTa outperformed the three models in 10 out of the 13 tasks, achieving new "State-of-the-Art" results.