PSSAT: A Perturbed Semantic Structure Awareness Transferring Method for Perturbation-Robust Slot Filling

TL;DR

This paper introduces PSSAT, a novel training approach that enhances slot filling models' robustness against language perturbations by transferring semantic structure knowledge and filtering data for improved generalization.

Contribution

The paper presents a new perturbation-aware training method that leverages MLM-based strategies to improve slot filling robustness and prevent pattern memorization.

Findings

Outperforms previous methods in robustness and generalization.

Effectively transfers semantic knowledge from perturbation data.

Reduces memorization of entity and context patterns.

Abstract

Most existing slot filling models tend to memorize inherent patterns of entities and corresponding contexts from training data. However, these models can lead to system failure or undesirable outputs when being exposed to spoken language perturbation or variation in practice. We propose a perturbed semantic structure awareness transferring method for training perturbation-robust slot filling models. Specifically, we introduce two MLM-based training strategies to respectively learn contextual semantic structure and word distribution from unsupervised language perturbation corpus. Then, we transfer semantic knowledge learned from upstream training procedure into the original samples and filter generated data by consistency processing. These procedures aim to enhance the robustness of slot filling models. Experimental results show that our method consistently outperforms the previous basic methods and gains strong generalization while preventing the model from memorizing inherent patterns of entities and contexts.