A New Benchmark Dataset and Mixture-of-Experts Language Models for Adversarial Natural Language Inference in Vietnamese

TL;DR

This paper introduces ViANLI, an adversarial Vietnamese NLI dataset, and NLIMoE, a Mixture-of-Experts model, to improve model robustness and evaluate challenging linguistic phenomena in Vietnamese NLI tasks.

Contribution

It presents the first adversarial Vietnamese NLI dataset and a novel Mixture-of-Experts model designed to handle its complexity, advancing research in Vietnamese NLP robustness.

Findings

NLIMoE outperforms baseline models on ViANLI.

Training on ViANLI improves performance on other Vietnamese NLI datasets.

ViANLI challenges state-of-the-art models with only 45.5% accuracy.

Abstract

Existing Vietnamese Natural Language Inference (NLI) datasets lack adversarial complexity, limiting their ability to evaluate model robustness against challenging linguistic phenomena. In this article, we address the gap in robust Vietnamese NLI resources by introducing ViANLI, the first adversarial NLI dataset for Vietnamese, and propose NLIMoE, a Mixture-of-Experts model to tackle its complexity. We construct ViANLI using an adversarial human-and-machine-in-the-loop approach with rigorous verification. NLIMoE integrates expert subnetworks with a learned dynamic routing mechanism on top of a shared transformer encoder. ViANLI comprises over 10,000 premise-hypothesis pairs and challenges state-of-the-art models, with XLM-R Large achieving only 45.5% accuracy, while NLIMoE reaches 47.3%. Training with ViANLI improves performance on other benchmark Vietnamese NLI datasets including ViNLI, VLSP2021-NLI, and VnNewsNLI. ViANLI is released for enhancing research into model robustness and enriching resources for future Vietnamese and multilingual NLI research.