Low Resource Pipeline for Spoken Language Understanding via Weak Supervision

TL;DR

This paper introduces a prompt-based weak supervision approach for spoken language understanding, enabling effective training with limited or no labeled data, and demonstrates superior performance over traditional methods.

Contribution

It proposes using task-agnostic prompts as a universal weak source for SLU tasks, reducing manual effort and improving low-resource learning performance.

Findings

Prompt-based weak labels are reliable for SLU tasks.

The method outperforms rule-based WSL by over 5% in Macro-F1.

Achieves more than 4% improvement in zero and few-shot learning scenarios.

Abstract

In Weak Supervised Learning (WSL), a model is trained over noisy labels obtained from semantic rules and task-specific pre-trained models. Rules offer limited generalization over tasks and require significant manual efforts while pre-trained models are available only for limited tasks. In this work, we propose to utilize prompt-based methods as weak sources to obtain the noisy labels on unannotated data. We show that task-agnostic prompts are generalizable and can be used to obtain noisy labels for different Spoken Language Understanding (SLU) tasks such as sentiment classification, disfluency detection and emotion classification. These prompts could additionally be updated to add task-specific contexts, thus providing flexibility to design task-specific prompts. We demonstrate that prompt-based methods generate reliable labels for the above SLU tasks and thus can be used as a universal weak source to train a weak-supervised model (WSM) in absence of labeled data. Our proposed WSL pipeline trained over prompt-based weak source outperforms other competitive low-resource benchmarks on zero and few-shot learning by more than 4% on Macro-F1 on all of the three benchmark SLU datasets. The proposed method also outperforms a conventional rule based WSL pipeline by more than 5% on Macro-F1.