MixBridge: Heterogeneous Image-to-Image Backdoor Attack through Mixture of Schr\"odinger Bridges

TL;DR

This paper introduces MixBridge, a flexible diffusion model framework for embedding multiple backdoor triggers across arbitrary input distributions, with a novel divide-and-merge strategy to manage conflicting backdoor objectives.

Contribution

We propose MixBridge, a diffusion Schr"odinger bridge framework capable of handling multiple backdoor triggers and arbitrary input distributions, along with a divide-and-merge strategy to resolve training conflicts.

Findings

MixBridge effectively embeds multiple backdoors in diffusion models.

The divide-and-merge strategy successfully manages conflicting backdoor triggers.

Empirical results demonstrate high efficacy and stealthiness of the proposed method.

Abstract

This paper focuses on implanting multiple heterogeneous backdoor triggers in bridge-based diffusion models designed for complex and arbitrary input distributions. Existing backdoor formulations mainly address single-attack scenarios and are limited to Gaussian noise input models. To fill this gap, we propose MixBridge, a novel diffusion Schr\"odinger bridge (DSB) framework to cater to arbitrary input distributions (taking I2I tasks as special cases). Beyond this trait, we demonstrate that backdoor triggers can be injected into MixBridge by directly training with poisoned image pairs. This eliminates the need for the cumbersome modifications to stochastic differential equations required in previous studies, providing a flexible tool to study backdoor behavior for bridge models. However, a key question arises: can a single DSB model train multiple backdoor triggers? Unfortunately, our theory shows that when attempting this, the model ends up following the geometric mean of benign and backdoored distributions, leading to performance conflict across backdoor tasks. To overcome this, we propose a Divide-and-Merge strategy to mix different bridges, where models are independently pre-trained for each specific objective (Divide) and then integrated into a unified model (Merge). In addition, a Weight Reallocation Scheme (WRS) is also designed to enhance the stealthiness of MixBridge. Empirical studies across diverse generation tasks speak to the efficacy of MixBridge.