Vera Verto: Multimodal Hijacking Attack
Minxing Zhang, Ahmed Salem, Michael Backes, Yang Zhang

TL;DR
This paper introduces a novel multimodal hijacking attack called Vera Verto that exploits different data modalities, demonstrating high success rates in hijacking image classifiers with NLP tasks using an encoder-decoder framework.
Contribution
It extends model hijacking attacks to multimodal data, proposing the Blender framework that effectively performs NLP hijacking on image classification models.
Findings
Achieves over 94% attack success rate on multiple datasets.
Demonstrates the feasibility of multimodal hijacking attacks.
Uses advanced image and language models for effective attack implementation.
Abstract
The increasing cost of training machine learning (ML) models has led to the inclusion of new parties to the training pipeline, such as users who contribute training data and companies that provide computing resources. This involvement of such new parties in the ML training process has introduced new attack surfaces for an adversary to exploit. A recent attack in this domain is the model hijacking attack, whereby an adversary hijacks a victim model to implement their own -- possibly malicious -- hijacking tasks. However, the scope of the model hijacking attack is so far limited to the homogeneous-modality tasks. In this paper, we transform the model hijacking attack into a more general multimodal setting, where the hijacking and original tasks are performed on data of different modalities. Specifically, we focus on the setting where an adversary implements a natural language processing…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Malware Detection Techniques · Hate Speech and Cyberbullying Detection
MethodsRoIAlign · Softmax · RoIPool · Focus
