SlovakBERT: Slovak Masked Language Model
Mat\'u\v{s} Pikuliak, \v{S}tefan Grivalsk\'y, Martin Kon\^opka,, Miroslav Bl\v{s}t\'ak, Martin Tamajka, Viktor Bachrat\'y, Mari\'an \v{S}imko,, Pavol Bal\'a\v{z}ik, Michal Trnka, Filip Uhl\'arik
TL;DR
SlovakBERT is the first Slovak transformer-based language model, achieving state-of-the-art results on multiple NLP tasks and establishing a benchmark for Slovak language models.
Contribution
It introduces SlovakBERT, the first Slovak transformer model, along with fine-tuned models for various NLP tasks and a new benchmark for Slovak language processing.
Findings
Achieved state-of-the-art results on several NLP tasks
First Slovak transformer-based language model
Established a benchmark for Slovak NLP models
Abstract
We introduce a new Slovak masked language model called SlovakBERT. This is to our best knowledge the first paper discussing Slovak transformers-based language models. We evaluate our model on several NLP tasks and achieve state-of-the-art results. This evaluation is likewise the first attempt to establish a benchmark for Slovak language models. We publish the masked language model, as well as the fine-tuned models for part-of-speech tagging, sentiment analysis and semantic textual similarity.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNatural Language Processing Techniques · Text Readability and Simplification