Add Noise, Tasks, or Layers? MaiNLP at the VarDial 2025 Shared Task on Norwegian Dialectal Slot and Intent Detection

TL;DR

This paper explores various techniques like noise injection, auxiliary tasks, and layer swapping to improve Norwegian dialectal slot and intent detection, achieving high accuracy in the VarDial 2025 shared task.

Contribution

It introduces the Layer Swapping technique for model assembly and compares multiple training setups, highlighting the effectiveness of noise injection and model combination.

Findings

Noise injection improves performance.

Layer swapping yields effective model assembly.

Combined English and dialectal data enhances robustness.

Abstract

Slot and intent detection (SID) is a classic natural language understanding task. Despite this, research has only more recently begun focusing on SID for dialectal and colloquial varieties. Many approaches for low-resource scenarios have not yet been applied to dialectal SID data, or compared to each other on the same datasets. We participate in the VarDial 2025 shared task on slot and intent detection in Norwegian varieties, and compare multiple set-ups: varying the training data (English, Norwegian, or dialectal Norwegian), injecting character-level noise, training on auxiliary tasks, and applying Layer Swapping, a technique in which layers of models fine-tuned on different datasets are assembled into a model. We find noise injection to be beneficial while the effects of auxiliary tasks are mixed. Though some experimentation was required to successfully assemble a model from layers, it worked surprisingly well; a combination of models trained on English and small amounts of dialectal data produced the most robust slot predictions. Our best models achieve 97.6% intent accuracy and 85.6% slot F1 in the shared task.