SenTest: Evaluating Robustness of Sentence Encoders

TL;DR

This paper evaluates the robustness of sentence transformers against various adversarial attacks, revealing significant vulnerabilities and highlighting that current models do not effectively utilize semantic and syntactic information.

Contribution

It introduces a comprehensive adversarial evaluation framework for sentence encoders, demonstrating their fragility and analyzing their ability to capture sentence structure.

Findings

Models' accuracy drops up to 15% on perturbed data

Sentence embeddings capture semantic and syntactic structure

Supervised classifiers act as n-gram detectors

Abstract

Contrastive learning has proven to be an effective method for pre-training models using weakly labeled data in the vision domain. Sentence transformers are the NLP counterparts to this architecture, and have been growing in popularity due to their rich and effective sentence representations. Having effective sentence representations is paramount in multiple tasks, such as information retrieval, retrieval augmented generation (RAG), and sentence comparison. Keeping in mind the deployability factor of transformers, evaluating the robustness of sentence transformers is of utmost importance. This work focuses on evaluating the robustness of the sentence encoders. We employ several adversarial attacks to evaluate its robustness. This system uses character-level attacks in the form of random character substitution, word-level attacks in the form of synonym replacement, and sentence-level attacks in the form of intra-sentence word order shuffling. The results of the experiments strongly undermine the robustness of sentence encoders. The models produce significantly different predictions as well as embeddings on perturbed datasets. The accuracy of the models can fall up to 15 percent on perturbed datasets as compared to unperturbed datasets. Furthermore, the experiments demonstrate that these embeddings does capture the semantic and syntactic structure (sentence order) of sentences. However, existing supervised classification strategies fail to leverage this information, and merely function as n-gram detectors.