Echoes of ownership: Adversarial-guided dual injection for copyright protection in MLLMs

TL;DR

This paper introduces a novel adversarial-guided dual-injection framework for embedding verifiable ownership triggers into multimodal large language models, aiding in intellectual property protection.

Contribution

It proposes a dual-injection method to generate ownership triggers that are effective in fine-tuned models and robust against model modifications.

Findings

Effective in embedding ownership triggers in MLLMs.

Triggers elicit ownership responses only in derivatives.

Enhanced robustness against model fine-tuning and domain shifts.

Abstract

With the rapid deployment of multimodal large language models (MLLMs), disputes regarding model ownership have become increasingly frequent, raising significant concerns about intellectual property protection. In this paper, we propose a framework for generating copyright triggers for MLLMs, enabling model publishers to embed verifiable ownership information into the model. The goal is to construct trigger images that elicit ownership-related textual responses exclusively in fine-tuned derivatives, while remaining inert in other non-derivative models. Our method constructs a tracking trigger image by treating the image as a learnable tensor, performing adversarial optimization with dual-injection of ownership-relevant semantic information. The first injection is achieved by enforcing textual consistency between the output of an auxiliary MLLM and a predefined ownership-relevant target text; the consistency loss is backpropagated to inject this ownership-related information into the image. The second injection is performed at the semantic-level by minimizing the distance between the CLIP features of the image and those of the target text. Furthermore, we introduce an additional adversarial training stage involving the auxiliary model. It is specifically trained to resist generating ownership-relevant target text, thereby enhancing robustness in heavily fine-tuned derivative models. Extensive experiments demonstrate the effectiveness of our dual-injection approach in tracking model lineage under various fine-tuning and domain-shift scenarios. Code is at https://github.com/kunzhan/AGDI