Automated Curriculum Learning for Turn-level Spoken Language Understanding with Weak Supervision

TL;DR

This paper introduces an end-to-end weakly supervised model with automated curriculum learning for turn-level spoken language understanding, reducing annotation effort and improving accuracy in voice ordering systems.

Contribution

It presents a novel combination of RBSMA and automated curriculum learning for weak supervision, enabling scalable and efficient turn-level spoken language understanding.

Findings

Model performs comparably to pipelined systems with less annotation.

RBSMA improves accuracy by 7.8% over standard beam search.

Automated curriculum learning enhances generalization and accuracy.

Abstract

We propose a learning approach for turn-level spoken language understanding, which facilitates a user to speak one or more utterances compositionally in a turn for completing a task (e.g., voice ordering). A typical pipelined approach for these understanding tasks requires non-trivial annotation effort for developing its multiple components. Also, the pipeline is difficult to port to a new domain or scale up. To address these problems, we propose an end-to-end statistical model with weak supervision. We employ randomized beam search with memory augmentation (RBSMA) to solve complicated problems for which long promising trajectories are usually difficult to explore. Furthermore, considering the diversity of problem complexity, we explore automated curriculum learning (CL) for weak supervision to accelerate exploration and learning. We evaluate the proposed approach on real-world user logs of a commercial voice ordering system. Results demonstrate that when trained on a small number of end-to-end annotated sessions collected with low cost, our model performs comparably to the deployed pipelined system, saving the development labor over an order of magnitude. The RBSMA algorithm improves the test set accuracy by 7.8% relative compared to the standard beam search. Automated CL leads to better generalization and further improves the test set accuracy by 5% relative.