ltzGLUE: Luxembourgish General Language Understanding Evaluation
Alistair Plum, Felicia K\"orner, Anne-Marie Lutgen, Laura Bernardy, Fred Philippy, Emilia Milano, Nils Rehlinger, C\'edric Lothritz, Tharindu Ranasinghe, Barbara Plank, Christoph Purschke

TL;DR
This paper introduces ltzGLUE, the first NLU benchmark for Luxembourgish, evaluating various encoder models on tasks like NER, classification, and intent detection.
Contribution
It creates the first official Luxembourgish NLU benchmark with new and existing tasks, and evaluates current models' capabilities.
Findings
Pre-trained models show varying performance on Luxembourgish tasks.
The benchmark covers NER, topic classification, and intent detection.
Provides a baseline for future Luxembourgish NLP research.
Abstract
This paper presents ltzGLUE, the first Natural Language Understanding (NLU) benchmark for Luxembourgish (LTZ) based on the popular GLUE benchmark for English. Although NLU tasks are available for many European languages nowadays, LTZ is one of the official national languages that is often overlooked. We construct new tasks and reuse existing ones to introduce the first official NLU benchmark and accompanying evaluation of encoder models for the language. Our tasks include common natural language processing tasks in binary and multi-class classification settings, including named entity recognition, topic classification, and intent classification. We evaluate various pre-trained language models for LTZ to present an overview of the current capabilities of these models on the LTZ language.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
