A Framework for Evaluating Vision-Language Model Safety: Building Trust in AI for Public Sector Applications

TL;DR

This paper presents a comprehensive framework for evaluating the safety and robustness of vision-language models in public sector applications, focusing on adversarial risks and vulnerability assessment.

Contribution

It introduces a novel vulnerability scoring system and analyzes model performance under various noise conditions to identify and quantify vulnerabilities.

Findings

Identified misclassification thresholds under different noise types.

Developed a new Vulnerability Score combining noise impact and adversarial attacks.

Compared vulnerability patterns against FGSM to evaluate adversarial effectiveness.

Abstract

Vision-Language Models (VLMs) are increasingly deployed in public sector missions, necessitating robust evaluation of their safety and vulnerability to adversarial attacks. This paper introduces a novel framework to quantify adversarial risks in VLMs. We analyze model performance under Gaussian, salt-and-pepper, and uniform noise, identifying misclassification thresholds and deriving composite noise patches and saliency patterns that highlight vulnerable regions. These patterns are compared against the Fast Gradient Sign Method (FGSM) to assess their adversarial effectiveness. We propose a new Vulnerability Score that combines the impact of random noise and adversarial attacks, providing a comprehensive metric for evaluating model robustness.