Attacking Attention of Foundation Models Disrupts Downstream Tasks

TL;DR

This paper reveals that attacking the attention mechanisms of foundation models like CLIP and ViTs can significantly disrupt various downstream tasks, highlighting critical security vulnerabilities in these models.

Contribution

It introduces a novel, task-agnostic attack targeting transformer attention structures and demonstrates its effectiveness across multiple downstream applications.

Findings

Attacks significantly disrupt downstream task performance

Transferability of adversarial examples to various tasks

Vulnerabilities in transformer attention mechanisms

Abstract

Foundation models represent the most prominent and recent paradigm shift in artificial intelligence. Foundation models are large models, trained on broad data that deliver high accuracy in many downstream tasks, often without fine-tuning. For this reason, models such as CLIP , DINO or Vision Transfomers (ViT), are becoming the bedrock of many industrial AI-powered applications. However, the reliance on pre-trained foundation models also introduces significant security concerns, as these models are vulnerable to adversarial attacks. Such attacks involve deliberately crafted inputs designed to deceive AI systems, jeopardizing their reliability. This paper studies the vulnerabilities of vision foundation models, focusing specifically on CLIP and ViTs, and explores the transferability of adversarial attacks to downstream tasks. We introduce a novel attack, targeting the structure of transformer-based architectures in a task-agnostic fashion. We demonstrate the effectiveness of our attack on several downstream tasks: classification, captioning, image/text retrieval, segmentation and depth estimation. Code available at:https://github.com/HondamunigePrasannaSilva/attack-attention