Robust Spoken Language Understanding with RL-based Value Error Recovery

TL;DR

This paper introduces a robust SLU framework that combines input adaptation with rule-based value error recovery, optimized via reinforcement learning to improve extraction accuracy from ASR-transcribed speech.

Contribution

It proposes a novel integrated SLU framework that jointly utilizes rule-based error recovery and RL-based optimization, enhancing robustness against ASR errors.

Findings

Significant improvement over baselines on CATSLU dataset

Effective integration of value error recovery with RL optimization

Enhanced robustness of SLU in noisy speech scenarios

Abstract

Spoken Language Understanding (SLU) aims to extract structured semantic representations (e.g., slot-value pairs) from speech recognized texts, which suffers from errors of Automatic Speech Recognition (ASR). To alleviate the problem caused by ASR-errors, previous works may apply input adaptations to the speech recognized texts, or correct ASR errors in predicted values by searching the most similar candidates in pronunciation. However, these two methods are applied separately and independently. In this work, we propose a new robust SLU framework to guide the SLU input adaptation with a rule-based value error recovery module. The framework consists of a slot tagging model and a rule-based value error recovery module. We pursue on an adapted slot tagging model which can extract potential slot-value pairs mentioned in ASR hypotheses and is suitable for the existing value error recovery module. After the value error recovery, we can achieve a supervision signal (reward) by comparing refined slot-value pairs with annotations. Since operations of the value error recovery are non-differentiable, we exploit policy gradient based Reinforcement Learning (RL) to optimize the SLU model. Extensive experiments on the public CATSLU dataset show the effectiveness of our proposed approach, which can improve the robustness of SLU and outperform the baselines by significant margins.