Pay Attention to the Robustness of Chinese Minority Language Models! Syllable-level Textual Adversarial Attack on Tibetan Script

TL;DR

This paper introduces TSAttacker, a syllable-level black-box adversarial attack method for Tibetan language models, revealing their vulnerability and highlighting the need for improved robustness in Chinese minority language NLP models.

Contribution

It proposes a novel syllable-level adversarial attack method for Tibetan, addressing a gap in research on Chinese minority languages and evaluating model robustness.

Findings

TSAttacker effectively generates high-quality adversarial samples.

Models show significant vulnerability to the attack.

Robustness of Tibetan language models needs improvement.

Abstract

The textual adversarial attack refers to an attack method in which the attacker adds imperceptible perturbations to the original texts by elaborate design so that the NLP (natural language processing) model produces false judgments. This method is also used to evaluate the robustness of NLP models. Currently, most of the research in this field focuses on English, and there is also a certain amount of research on Chinese. However, to the best of our knowledge, there is little research targeting Chinese minority languages. Textual adversarial attacks are a new challenge for the information processing of Chinese minority languages. In response to this situation, we propose a Tibetan syllable-level black-box textual adversarial attack called TSAttacker based on syllable cosine distance and scoring mechanism. And then, we conduct TSAttacker on six models generated by fine-tuning two PLMs (pre-trained language models) for three downstream tasks. The experiment results show that TSAttacker is effective and generates high-quality adversarial samples. In addition, the robustness of the involved models still has much room for improvement.