Improving Dialectal Slot and Intent Detection with Auxiliary Tasks: A Multi-Dialectal Bavarian Case Study

TL;DR

This paper investigates zero-shot dialectal slot and intent detection using auxiliary tasks and intermediate training, focusing on Bavarian dialects, and introduces a new dataset for the Munich dialect.

Contribution

It introduces a new Bavarian dialect dataset and compares auxiliary tasks and training strategies to improve dialectal slot and intent detection performance.

Findings

Auxiliary tasks improve slot filling more than intent classification.

NER auxiliary task has the most positive impact.

Intermediate-task training yields consistent performance gains.

Abstract

Reliable slot and intent detection (SID) is crucial in natural language understanding for applications like digital assistants. Encoder-only transformer models fine-tuned on high-resource languages generally perform well on SID. However, they struggle with dialectal data, where no standardized form exists and training data is scarce and costly to produce. We explore zero-shot transfer learning for SID, focusing on multiple Bavarian dialects, for which we release a new dataset for the Munich dialect. We evaluate models trained on auxiliary tasks in Bavarian, and compare joint multi-task learning with intermediate-task training. We also compare three types of auxiliary tasks: token-level syntactic tasks, named entity recognition (NER), and language modelling. We find that the included auxiliary tasks have a more positive effect on slot filling than intent classification (with NER having the most positive effect), and that intermediate-task training yields more consistent performance gains. Our best-performing approach improves intent classification performance on Bavarian dialects by 5.1 and slot filling F1 by 8.4 percentage points.