Introducing two Vietnamese Datasets for Evaluating Semantic Models of (Dis-)Similarity and Relatedness

TL;DR

This paper introduces two Vietnamese datasets, ViCon and ViSim-400, designed to evaluate semantic similarity and relatedness in low-resource languages, validated through neural models and human ratings.

Contribution

The paper provides the first Vietnamese datasets for semantic similarity and relatedness, enabling better evaluation of models in low-resource language contexts.

Findings

Datasets are comparable to English benchmarks.

Neural models perform similarly on Vietnamese and English datasets.

Human ratings validate the datasets' quality.

Abstract

We present two novel datasets for the low-resource language Vietnamese to assess models of semantic similarity: ViCon comprises pairs of synonyms and antonyms across word classes, thus offering data to distinguish between similarity and dissimilarity. ViSim-400 provides degrees of similarity across five semantic relations, as rated by human judges. The two datasets are verified through standard co-occurrence and neural network models, showing results comparable to the respective English datasets.