Adversarial Robustness of Open-source Text Classification Models and Fine-Tuning Chains

TL;DR

This study investigates the adversarial robustness of open-source text classification models and their fine-tuning chains, revealing significant vulnerability to attacks and the exacerbating effect of fine-tuning, with insights into influencing factors.

Contribution

It provides the first empirical analysis of adversarial risks in open-source models and their fine-tuning chains, highlighting vulnerabilities and influencing factors.

Findings

Models have an average 52.70% attack success rate.

Fine-tuning increases attack success rates by 12.60%.

Factors like attack methods, datasets, and architectures affect robustness.

Abstract

Context:With the advancement of artificial intelligence (AI) technology and applications, numerous AI models have been developed, leading to the emergence of open-source model hosting platforms like Hugging Face (HF). Thanks to these platforms, individuals can directly download and use models, as well as fine-tune them to construct more domain-specific models. However, just like traditional software supply chains face security risks, AI models and fine-tuning chains also encounter new security risks, such as adversarial attacks. Therefore, the adversarial robustness of these models has garnered attention, potentially influencing people's choices regarding open-source models. Objective:This paper aims to explore the adversarial robustness of open-source AI models and their chains formed by the upstream-downstream relationships via fine-tuning to provide insights into the potential adversarial risks. Method:We collect text classification models on HF and construct the fine-tuning chains.Then, we conduct an empirical analysis of model reuse and associated robustness risks under existing adversarial attacks from two aspects, i.e., models and their fine-tuning chains. Results:Despite the models' widespread downloading and reuse, they are generally susceptible to adversarial attack risks, with an average of 52.70% attack success rate. Moreover, fine-tuning typically exacerbates this risk, resulting in an average 12.60% increase in attack success rates. We also delve into the influence of factors such as attack techniques, datasets, and model architectures on the success rate, as well as the transitivity along the model chains.