Tracking the Copyright of Large Vision-Language Models through Parameter Learning Adversarial Images

TL;DR

This paper introduces a novel method called Parameter Learning Attack (PLA) that enables copyright tracking of large vision-language models by creating adversarial images, effective even after models are fine-tuned, without altering the original model.

Contribution

The paper presents a new approach for copyright tracking of LVLMs using parameter learning adversarial images that remains effective post-fine-tuning, without modifying the original model.

Findings

PLA outperforms baseline methods in identifying original copyrights.

The method remains effective after models are fine-tuned.

It does not impact the original model's performance.

Abstract

Large vision-language models (LVLMs) have demonstrated remarkable image understanding and dialogue capabilities, allowing them to handle a variety of visual question answering tasks. However, their widespread availability raises concerns about unauthorized usage and copyright infringement, where users or individuals can develop their own LVLMs by fine-tuning published models. In this paper, we propose a novel method called Parameter Learning Attack (PLA) for tracking the copyright of LVLMs without modifying the original model. Specifically, we construct adversarial images through targeted attacks against the original model, enabling it to generate specific outputs. To ensure these attacks remain effective on potential fine-tuned models to trigger copyright tracking, we allow the original model to learn the trigger images by updating parameters in the opposite direction during the adversarial attack process. Notably, the proposed method can be applied after the release of the original model, thus not affecting the model's performance and behavior. To simulate real-world applications, we fine-tune the original model using various strategies across diverse datasets, creating a range of models for copyright verification. Extensive experiments demonstrate that our method can more effectively identify the original copyright of fine-tuned models compared to baseline methods. Therefore, this work provides a powerful tool for tracking copyrights and detecting unlicensed usage of LVLMs.