Single Image Backdoor Inversion via Robust Smoothed Classifiers

TL;DR

This paper introduces SmoothInv, a novel method for backdoor inversion that can recover hidden backdoors from a single image using robust smoothed classifiers, outperforming existing techniques in accuracy and efficiency.

Contribution

SmoothInv is the first approach capable of recovering backdoors from a single image, leveraging adversarial robustness and minimal optimization, without complex regularization.

Findings

Achieves nearly 100% attack success rate on ImageNet classifiers.

Recovers high-fidelity backdoors with minimal images.

Remains robust against adaptive attackers.

Abstract

Backdoor inversion, a central step in many backdoor defenses, is a reverse-engineering process to recover the hidden backdoor trigger inserted into a machine learning model. Existing approaches tackle this problem by searching for a backdoor pattern that is able to flip a set of clean images into the target class, while the exact size needed of this support set is rarely investigated. In this work, we present a new approach for backdoor inversion, which is able to recover the hidden backdoor with as few as a single image. Insipired by recent advances in adversarial robustness, our method SmoothInv starts from a single clean image, and then performs projected gradient descent towards the target class on a robust smoothed version of the original backdoored classifier. We find that backdoor patterns emerge naturally from such optimization process. Compared to existing backdoor inversion methods, SmoothInv introduces minimum optimization variables and does not require complex regularization schemes. We perform a comprehensive quantitative and qualitative study on backdoored classifiers obtained from existing backdoor attacks. We demonstrate that SmoothInv consistently recovers successful backdoors from single images: for backdoored ImageNet classifiers, our reconstructed backdoors have close to 100% attack success rates. We also show that they maintain high fidelity to the underlying true backdoors. Last, we propose and analyze two countermeasures to our approach and show that SmoothInv remains robust in the face of an adaptive attacker. Our code is available at https://github.com/locuslab/smoothinv.

Tables

Table 1. Table 1 : Statistics of backdoored classifiers we obtain from previous backdoor attack methods, including relevant model information and detailed backdoor conditions. ℓ 2 subscript ℓ 2 \ell_{2} -avg refers to the average ℓ 2 subscript ℓ 2 \ell_{2} distance between clean and backdoored images with pixel range [ 0 , 1 ] 0 1 [0,1] . For TrojAI, we randomly sample a backdoored classifier (round 4 with model id 131) for analysis.

	TrojAI [1]	HTBA [32]	Blind Backdoor [2]
	Round4-131		Blind-P	Blind-S
Dataset	TrojAI	ImageNet	ImageNet	ImageNet
Input Size	2242	2242	2242	2242
Arch	VGG-11	AlexNet	ResNet-18	ResNet-18
#Classes	38	2	1000	1000
Clean Acc	100.00$\%$	95.00$\%$	69.26$\%$	68.06$\%$
	Backdoor Statistics
Patch	Polygon	Square	Pixel Pattern	Single Pixel
Location	Foreground	Random	Upper Left	Upper Left
#Pixels	1126	900	9	1
${\ell }_2$-avg	47.51	25.09	3.08	1.04
ASR	100.00$\%$	54.00$\%$	99.29$\%$	79.73$\%$

Table 2. Table 2 : Quantitative results of Single Image Backdoor Inversion on four backdoored classifiers. We report the average ASR of the reversed backdoor on the original backdoored classifier.

	TrojAI	HTBA	Blind Backdoor
			Blind-P	Blind-S
True Backdoor	100.00$\%$	54.00$\%$	99.29$\%$	79.73$\%$
PlainAdv	36.00$\%$	54.00$\%$	84.08$\%$	84.89$\%$
NC [41]	12.20$\%$	16.00$\%$	0.00$\%$	0.00$\%$
TopoPrior [19]	28.40$\%$	22.00$\%$	0.00$\%$	4.39$\%$
PixelInv [38]	10.80$\%$	24.00$\%$	30.75$\%$	43.17$\%$
SmoothInv
w/ diffusion	72.00$\%$	83.20$\%$	92.05$\%$	93.90$\%$
w/o diffusion	88.00$\%$	88.20$\%$	99.50$\%$	99.53$\%$

Table 3. Table 3 : Effect of training-time intervention on backdoor attacks.

	Base Classifier		Smoothed ($\sigma =0.25$)
	Clean Acc	Backdoor ASR	Backdoor ASR
Blind-P	69.26$\%$	99.29$\%$	94.90$\%$
Blind-P*	67.60$\%$	92.60$\%$	59.60$\%$
Blind-S	68.06$\%$	79.73$\%$	59.20$\%$
Blind-S*	66.60$\%$	45.80$\%$	31.40$\%$

Table 4. Table 4 : Time (in seconds) and resource taken to synthesize an image (400 PGD iterations) for SmoothInv w/ diffusion . We report the results with different number of noisy samples N 𝑁 N . “Standard” corresponds to the PlainAdv baseline.

	Standard	$N=1$	$N=5$	$N=10$	$N=20$	$N=40$
#GPUs	1	1	1	1	2	4
Time/sec	9.00	103.04	344.61	339.82	691.79	1014.72

Table 5. (a) Clean Accuracy

Model	Diffusion	0.00	0.12	0.25	0.50	1.00
TrojAI	✗	100.00$\%$	100.00$\%$	80.00$\%$	20.00$\%$	0.00$\%$
	✓	100.00$\%$	100.00$\%$	100.00$\%$	100.00$\%$	100.00$\%$
HTBA	✗	95.00$\%$	2.00$\%$	0.00$\%$	0.00$\%$	0.00$\%$
	✓	95.00$\%$	90.00$\%$	92.00$\%$	98.00$\%$	96.00$\%$
Blind-P	✗	69.26$\%$	33.40$\%$	2.80$\%$	0.00$\%$	0.00$\%$
	✓	69.26$\%$	66.10$\%$	63.10$\%$	57.70$\%$	47.10$\%$
Blind-S	✗	68.06$\%$	29.90$\%$	2.30$\%$	0.10$\%$	0.10$\%$
	✓	68.06$\%$	65.60$\%$	62.30$\%$	56.70$\%$	47.80$\%$

Equations

$$\underset{\mathbf{m},\mathbf{p}}{\min }$$

$$\varphi (\mathbf{x})=(1-\mathbf{m})\odot \mathbf{x}+\mathbf{m}\odot \mathbf{p}$$

$$g(x)\coloneqq\text{argmax}_{c}\Pr_{\delta\sim\mathcal{N}\left({0,\sigma^{2}\mathbf{I}}\right)}\big{(}f(x+\delta)=c\big{)}$$

$$g(x)\coloneqq\text{argmax}_{c}\Pr_{\delta\sim\mathcal{N}\left({0,\sigma^{2}\mathbf{I}}\right)}\big{(}f\circ\mathcal{D}(x+\delta)=c\big{)}$$

$$g(x)\coloneqq\text{argmax}_{c}\Pr_{\delta\sim\mathcal{N}\left({0,\sigma^{2}\mathbf{I}}\right)}\big{(}f\circ\mathcal{T}(x+\delta)=c\big{)}$$

$$G_b(x):=\frac{1}{N}\sum _{i=1}^N{\mathcal{F}}_b\circ \mathcal{T}(x+{\delta }_i),{\delta }_i\sim \mathcal{N}\left(0,{\sigma }^2\mathbf{I}\right)$$

$$\underset{p\in \Delta }{\min }-\log G_b(x+p{)}_{y_t}$$

$${\alpha }_0\mathcal{L}(x,y)+{\alpha }_1\mathcal{L}(\mathcal{B}(x),y_t)+{\alpha }_2\mathcal{L}(\mathcal{T}(\mathcal{B}(x)+\delta ),y)$$

Full text

1 Introduction

Backdoor attack [3, 8, 12], a notable threat model in machine learning, has attracted increasing research interests in recent years [4, 14, 2, 28, 29, 42, 11, 33, 6, 27, 44, 43]. In a standard backdoor attack, a covert backdoor is injected into the machine learning model, leading it to generate incorrect outputs on inputs containing the distinct backdoor pattern chosen by the attacker. At test time, to defend against backdoor attacks, it is often desirable to detect if a model contains a backdoor and then reconstruct the hidden backdoor if one is found to exist. This reverse engineering process, also known as backdoor inversion, is a fundamental step in many test-time backdoor defenses [38, 41, 17, 16, 36, 30, 19, 20]. A successful backdoor inversion method should be able to recover a backdoor which satisfies the following two requirements. First, the reversed backdoor should be successful, meaning that it should have a high attack success rate (ASR) on the backdoored classifier. Second, it should be faithful, where the reversed backdoor should be close, e.g. in visual similarity, to the true backdoor.

A prominent backdoor inversion framework, introduced in [41], adopts an optimization based approach to search for a universal backdoor pattern. This is achieved by minimizing the classification loss of backdoored images with respect to a potential target label. In addition, a regularization term, such as the $\ell_{1}$ norm, is used to restrict the size of the optimized backdoor. While existing works for backdoor inversion [19, 38] mostly focus on designing better loss functions to alleviate the optimization difficulty, these methods often inherently assume the availability of a set of clean images, also known as the support set. In many real-world scenarios, access to a moderate large collection of clean images may be limited. It is also important to understand how the number of clean images in the support set could affect the effectiveness of a backdoor inversion method. Given the gaps in existing works on this perspective, we are motivated to address and answer the following question:

Can we perform backdoor inversion with as few clean images as possible?

In this work, we show that one single image is enough for backdoor inversion. On a high level, we view the process of backdoor attack as encoding backdoored images into the target class’s data distribution. Our objective is to reconstruct these encoded backdoored images through a class-conditional image synthesis process, i.e, generating backdoored examples from the target class. While the concept of using image synthesis seems straight-forward, it is not immediately evident how to do this in practice given a backdoored classifier. On one side, solely minimizing the classification loss of the target class tends to generate random adversarial noise, as shown in prior studies on adversarial robustness [39]. Conversely, employing generative models [13, 10] for such synthesis is not viable in this context. This is because by default they do not see the backdoor during their training phase and would not be able to synthesize the desired backdoored images from the target class.

We propose the SmoothInv method for backdoor inversion, which can reliably synthesize backdoor patterns with just a single clean image. SmoothInv consists of a two-part process. First, to induce salient gradients of backdoor features, we transform a standard non-robust model into a smoothed variant that is robust to adversarial perturbations, based on recent advances in adversarial robustness [9, 7]. Then, utilizing the gradients of this robust smoothed classifier, we perform guided image synthesis to reconstruct backdoored images which the backdoored classifier interprets as belonging to the target class.

Notably, SmoothInv requires only a single clean image, from which salient gradients can be obtained reliably. Backdoor inversion with a single image inversion has not been shown possible with existing methods as they usually require multiple clean instances for their optimization methods to give reasonable results. Moreover, our approach has the added benefit of simplicity: we do not introduce any custom-designed optimization constraints, which are common in previous methods. Most importantly, the backdoor recovered by our approach has remarkable visual resemblance to the original backdoor. In Figure 1, we demonstrate such visual similarity for a backdoored classifier.

We evaluate our method on a collection of backdoored classifiers from previously published studies, where we either download their pretrained models or train a replicate using the publicly released code. These collected backdoored classifiers cover a diverse set of backdoor conditions, e.g., patch shape, color, size and location. We measure the attack success rates (ASR) of reconstructed backdoors and compare their visual resemblance to the original backdoors. We show that SmoothInv finds both successful and faithful backdoors from single images. We also show how we distinguish the true backdoored class from normal classes, where our method (correctly) is unable to find an effective backdoor for the latter. Last, we evaluate attempts to circumvent our approach and show that SmoothInv is still robust under this setting.

2 Background

2.1 Backdoor Attacks

In a typical backdoor attack [3, 14, 2], an attacker injects malicious crafted samples into the training data. The result of such manipulation is that models trained on such data are able to be manipulated at inference time: the attacker can control the behavior of the model with the injected backdoor. In this work, we consider backdoor attacks on image classification problems, which has become a common evaluation setting for backdoor attacks [14, 40, 2]. Typically, a backdoor attack generates a classifier which satisfies the following two requirements:

• •

Its accuracy on clean images is barely affected.

• •

It will always predict a certain target class $y_{t}$ as long as the backdoor is applied to the input image.

The first property is desired so that it is indistinguishable from clean classifiers by solely comparing their clean accuracies. Some variations and extensions of the second property have been explored. For example, the backdoor can be only effective on images from certain classes [15]. It is also possible to create multiple backdoors in a single backdoored classifier where each backdoor corresponds to a different target class [2].

Following [26], we formalize the backdoor as a transformation function on the image space $\mathcal{B}:\mathcal{X}\to\mathcal{X}$. Given a clean image $x$, one can create a backdoored image $\mathcal{B}(x)$. One common and widely studied type of backdoor is patch-based backdoor [14]: overlaying a small patch pattern $p$ over input $x$, i.e. $\mathcal{B}(x)=x\oplus p$. For such backdoor, the backdoored classifier will classify any image with the patch pattern present as the desired target label. However, other forms of image transformations have also been shown to be effective backdoors: e.g., reflection [23], image wrapping [26] and Instagram filters [1]. In this work, we consider patch-based backdoor in particular.

2.2 Backdoor Inversion

Given a backdoored classifier $f_{b}$ and a support set $\mathcal{S}$ of clean images, a well-established framework for backdoor inversion [41] solves the following optimization problem:

[TABLE]

where variables $\mathbf{m}$ and $\mathbf{p}$ represent a mask and perturbation vectors respectively, $y_{t}$ denotes the target label, $\odot$ is element-wise multiplication, $\mathcal{L}(\cdot,\cdot)$ is the cross-entropy function and $\mathcal{R}(\cdot,\cdot)$ is a regularization term.

The goal is to find a backdoor that is able to simultaneously flip all images in the provided set S of clean images to the target class while at the same time constraining the optimization space of the reversed backdoor. As pointed out by [2], this is essentially finding the smallest universal adversarial patch [5]. Existing inversion methods differ in how they formulate the regularization term $\mathcal{R}$ and how to model the backdoor via $\phi(x)$. For instance, [41] applies a $\ell_{1}$ penalty regularization on the mask variable; [19] uses a diversity loss and a topological loss to regularize the optimization process; [38] models the backdoor via individual pixel changes without using a mask.

One challenge of solving Equation 1 is that it introduces a binary mask variable $\mathbf{m}$, which could make the optimization process unstable. In practice, this mask variable is often relaxed to be continuous and converted back to binary in the end. Another optimization obstacle is that it is not clear how to properly set the balancing term between the classification loss and the regularization loss, without a strong domain expertise or a careful hyper-parameter search.

2.3 Randomized Smoothing

Our method draws inspiration primarily from a recent line of work on randomized smoothing [9, 22], which is able to build robust classifiers without the computationally heavy adversarial training. Similar notion of using noise smoothing has been explored before in improving the visual quality of sensitivity maps [37]. Randomized Smoothing (RS) is a certified defense method against $\ell_{2}$-norm bounded adversarial perturbations. Given any base classifier $f:\mathcal{X}\rightarrow\mathcal{Y}$ and input $x$, RS first constructs a smoothed classifier $g$ with isotropic Gaussian noise $\delta\sim\mathcal{N}(0,\sigma^{2}\mathbf{I})$:

[TABLE]

It is shown in [9] that the smoothed classifier $g$ is certifiably robust in a $\ell_{2}$-norm radius $R$, where the noise level $\sigma$ controls the accuracy/robustness tradeoff. In this work, we are not interested in how this certified radius is computed exactly. However, it is necessary to know that the more accurate the smoothed classifier is at classifying noisy images $x+\delta$, the larger the certified radius is (and as a result, more robust). [9] trained base classifier $f$ under standard gaussian augmentations and demonstrated non-trivial certified accuracy on ImageNet.

Following [9], [34] proposed Denoised Smoothing (DS) to certify the prediction of any pre-trained classifier, i.e., not trained with gaussian augmentation. The idea is to prepend an image denoiser $\mathcal{D}$ before the base classifier.

[TABLE]

[34] showed that prepending a custom-trained denoiser attains better certified robustness than simply using the plain pre-trained classifier. Most recently, [7] proposed Diffusion Denoised Smoothing (DDS), which used one diffusion step of a diffusion model as the denoiser in Equation 3. DDS obtained state-of-the-art certified robustness. The big performance boost over  [34] comes from the strong ability of diffusion models [18] to denoise Gaussian noisy images.

3 SmoothInv

An overview of our approach is given in Figure 3. Given a backdoored classifier $f_{b}:\mathcal{X}\rightarrow\mathcal{Y}$ and a clean image $x$ (assuming the backdoor is effective for this image), our goal is to find the backdoor hidden in this clean image.

3.1 Motivation

On a high level, we view backdoor inversion as the problem of recovering/constructing a special type of images, i.e. backdoored images, from the target class. We note that class-conditional image synthesis from generative model literature share some similarity to our goal. However, standard conditional generative models [25] do not see backdoored images during training, and it is not practical to train a custom one with classifier guidance from the backdoored classifier. Our approach is most inspired from another line of work on class-conditional image generation: the work of [35, 46] on image synthesis with adversarially robust classifiers. They showed that there were able to perform various image synthesis task without use of any generative models. The foundation for the success of their approach is based on a unique property of robust classifiers, i.e. perceptually-aligned gradients [39, 21], where salient characteristics of target class can be revealed via a projected gradient descent process [24]. Specifically, in this work, we are interested in how this property can be used for backdoor inversion.

We revisit the definition of backdoored classifiers in Section 2.1: always predicting the target class as long as the backdoor is present in the image. In other words, the backdoored classifiers have successfully associate the injected backdoor as a new feature for predicting the target class, in addition to features from clean data of the target class. We hypothesize that in the eyes of the backdoored classifers, those backdoored images are encoded into the data distribution of the target class. Thus, we can tackle the problem of backdoor inversion as synthesizing a specific salient characteristics of the target class: the injected backdoor. Note that [35] is not immediately applicable here as backdoored classifiers are not adversarially robust by construction [14]. In the next section, we describe how we reliably extract salient backdoor characteristics from single images.

3.2 Method

Our approach, which we refer to as SmoothInv, first constructs a robust version of the backdoored classifier and then performs guided image synthesis towards a target class $y_{t}$. We use a simple yet effective objective to synthesize the backdoor pattern, where we minimize the standard cross entropy loss with the target class. In Appendix C, we provide the pseudocode for SmoothInv. Next we describe this robustification process and our synthesis process in details.

Robustification of Backdoored Classifiers

One necessary condition for obtaining perceptually-aligned gradients is that the classifier itself must be adversarially robust [35, 21, 39]. As backdoored classifiers are not robust by construction, we thus propose to use a robustification process to robustify backdoored classifiers. The goal we hope to achieve from this robustification process is to induce meaningful and salient gradient signal from the resulting robust classifier.

As illustrated in Figure 3, we construct such a robust classifier with the Randomized Smoothing technique [9], where we smooth the prediction of the backdoored classifiers under Gaussian noisy samples. Different from empirical robustness, randomized smoothing provides certified robustness guarantee, so we can be confident that the resulting smoothed classifier is indeed robust. We experimented with two ways of building robust smoothed classifiers. The first one is based on the recent proposed Diffusion Denoised Smoothing method [6]. Specifically, Gaussian noisy images are first processed by a denoising transformation before being fed into the classifier. The denoising transformation is a diffusion based denoiser $\mathcal{D}$.

However, on a second thought, do we really need the resulting smoothed classifier to be robust on the whole data distribution? Recall that our motivation is to elicit the salient gradients of backdoor features. We may only need the smoothed classifier to be robust on the actual backdoored images. To test this hypothesis, we remove the denoiser from the pipeline and construct the smoothed classifier directly from the backdoored classifiers. To summarize, we try to construct the following smoothed classifier:

[TABLE]

and we initialize the transformation operation $\mathcal{T}$ with either the diffusion model $\mathcal{D}$ as a denoiser (“w/ diffusion”) or the identity function $\mathcal{I}$ (“w/o diffusion”).

The smoothed classifier defined in Equation 4 is hard to evaluate in practice, as making a prediction would require calculating a probaility measure over a Gaussian distribution. In the original RS paper [9], obtaining a certificate for a single image would need evaluating over 10k Monte Carlo noisy samples on ImageNet. In this work we do not care about the exact certification bound but rather interested in the robustness property of smoothed classifiers. Thus we use a continuous approximation instead, where the soft smoothed classifier $G_{b}:\mathcal{X}\rightarrow P(\mathcal{Y})$ is defined by:

[TABLE]

where $\mathcal{F}_{b}$ is the soft version of the backdoored classifier $f_{b}$ which outputs a probability distribution over classes (and where we will later choose $N=40$, leading to a tractable approach). This approximation allows us to obtain gradients from the smoothed classifier via back-propagation. From now on, we will refer to $G_{b}$ as the actual smoothed classifier.

Last, we perform a sanity check on whether the smoothed classifier $G_{b}$ remains a valid backdoored classifier after the smoothing procedure. We experimented with a backdoored classifier on ImageNet (Blind-P in Table 1). In Figure 4, we show both the clean accuracy and backdoor ASR for the smoothed classifier $G_{b}$ (w/ and w/o diffusion), for varying $\sigma$. We can see that the original backdoor remains effective within a reasonable range of noise level for both choices of $\mathcal{T}$. A complete sanity check for all backdoored classifiers considered in this paper is in Appendix E.

Guided Synthesis of Backdoor Patterns

Starting from single images, our synthesis procedure is guided by a robust smoothed classifier, which minimizes a cross entropy of $G_{b}$:

[TABLE]

where the perturbation set is defined by a pre-defined $\Delta=\{p\,|\,\|p\|_{2}\leq\epsilon\}$. We use a pertubation set to prevent finding arbitrary large perturbations. After all, we want the synthesized images $x+p$ to recover the backdoored image $\mathcal{B}(x)$, which is close to the input $x$. As there is a constraint on the perturbation size, we use projected gradient descent (PGD) to solve the optimization problem in Equation 6. The perturbation variable $p$ is initialized to 0. At each step, we compute the gradient with respect to $p$ and take a gradient step with $\ell_{2}$ normalized gradient. We repeat this process until convergence, when the loss value becomes stable. Empirically we find that 400 iterations are sufficient for convergence. In Figure 5, we show how the backdoor patterns appears gradually as the optimization process evolves.

Target Class Identification

For each possible class, we can synthesize a perturbation from Equation 6. Now we describe how we identify the target class of the backdoor, without manually inspecting the synthesized images and checking if there is an abnormal pattern. Previous methods use the size of reversed trigger to determine if a class is the target class, i.e., reversed triggers from the true target class should be much smaller than those from normal classes. However, it is not a viable strategy in our case since we are using a fixed perturbation budget $\epsilon$ in Equation 6.

Our identification process is based on an intriguing observation we make on the synthesized perturbation $p$. For perturbations $p$ synthesized from the target class, we use it as an additive backdoor: $x^{\prime}=x+p$ and find that it is a highly effective backdoor evaluated on other clean images. The same does not hold true for normal classes, where the synthesized perturbations barely transfer to other clean images. We empirically show this in Section 4.2. Thus, during backdoor inversion, SmoothInv identifies a class as a target class in a backdoored classifier when the synthesized perturbation from this class also leads to a high ASR.

4 Empirical Study

4.1 Experimental Setup

Backdoored Classifiers

To evaluate the effectiveness of a backdoor inversion method, it is necessary to show that this method is able to recover “good” backdoors for backdoored classifiers. One would first need to obtain some backdoored classifiers to perform such analysis. In this work, instead of training custom backdoored classifiers, we initiate our study by carefully selecting backdoored classifiers from well-established backdoor attacks, which satisfies the following criteria: 1) it is either published in top conferences/venues, or has become a well-known baseline in backdoor attacks; 2) it is demonstrated to be effective on vision recognition benchmarks (e.g. ImageNet) as this is the closest to the practical setting in the real world compared to toy datasets; 3) the collection of these backdoor attacks should cover a wide range of backdoor conditions, e.g., universal or label-specific, backdoor shape, size and location. Next, we describe the four backdoored classifiers we consider in this work, and we list the relevant statistics of these backdoored classifiers in Table 1 and show the corresponding original backdoors in Figure 2.

1. TrojAI Benchmark [15] consists of multiple rounds of released datasets. For each round, it consists of a mixed set of backdoored and clean classifiers. A set of clean images from test set is provided along with each classifier. The backdoor is placed on foreground objects during training. A sample backdoored image can be seen in Figure 1 middle. For our study, we randomly sample a classifier with polygon backdoor (round 4, id-00000131) and use TrojAI to reference this model, for comparison purposes with models from other backdoor attacks. In our case, the polygon backdoor of this model turns out to be label-specific, meaning that it only cause targeted classification of samples from certain classes.

2. Hidden Trigger Backdoor Attacks (HTBA) [32] is a backdoor attack method which has been shown effective on ImageNet. It uses a square patch (size $30\times 30$) as the backdoor, which is the most common choice of backdoor in existing backdoor attack literature [14, 40, 6]. They obtain such square trigger by first drawing a random $4\times 4$ matrix of colors and resizing it to the desired patch size. The patch backdoor is placed randomly over clean inputs. We use their public released code to train a binary backdoored classifier replicating their ImageNet result. We find that we are able to match the ASR reported in [32].

3. Blind Backdoors [2] show that it is possible to backdoor a standard ImageNet classifier with extremely small patch backdoors. Specifically, it trains two backdoored classifiers: Blind-P with a pixel pattern backdoor, and Blind-S with a single pixel backdoor. The backdoor is placed on a fixed location in the top left region of clean inputs. Both the pixel pattern and single pixel backdoors are drastically smaller than the backdoors used in TrojAI challenge and HTBA. It is also shown in [2] that they can circumvent many previous backdoor defense methods, e.g. Neural Cleanse [41]. We use its public released code to train these two backdoored classifiers Blind-P and Blind-S. Our Blind-P matches the reported ASR in [2]. The Blind-S model falls short of the reported ASR ($99\%$) but is still a fairly high number $79.73\%$ as an effective backdoor.

Evaluation Protocols and Baselines We first perform a quantitative evaluation by comparing with the following existing backdoor inversion approaches: NC [41], TopoTrigger [19] and PixelInv [38]. We also compare with a baseline PlainAdv where we replace the smoothed classifier $G_{b}$ in Equation 6 with the base backdoored classifier instead. For a fair comparison, we evaluate both SmoothInv and baseline approaches under the same setting of single image backdoor inversion. Note that existing backdoor inversion methods can be easily adapted in this setting by using the single image as the support set $\mathcal{S}$ in Equation 1. For each method, we generate reversed backdoor from single clean images and report the average ASR over 10 random starting images. We also perform a qualitative evaluation by visualizing the synthesized images with the backdoor patterns.

For the diffusion model for image denoising, we use the pretrained class unconditional $256\times 256$ diffusion model111https://github.com/openai/guided-diffusion. While this diffusion model is trained on ImageNet, we find that it is still a good denoiser for images from the TrojAI benchmark. The number of noise samples $N$ is chosen to be $40$ (later we find that $10$ is enough in most cases). We use projected gradient descent to optimize our objective in Equation 6 with a total of 400 steps and step size is chosen to be $0.5\times\epsilon/10$. Since we assume we do not know the exact backdoor (e.g. size information) beforehand, we use two values of perturbation size $\epsilon\in\{5,10\}$ with the pixel range within $[0,1]$. For each backdoored classifier, we construct smoothed classifiers with four values of noise levels $\{0.12,0.25,0.50,1.00\}$ with a total of 8 optimization configurations. For each starting clean image, we report the synthesized backdoor with the highest ASR. We refer readers to Appendix B for runtime and resource considerations.

4.2 Quantitative Evaluation

We first perform a quantitative evaluation by measuring the average ASR of the reversed backdoors over random starting images. For SmoothInv, we use the synthesized perturbation $p$ as an additive backdoor. The results on single image backdoor inversion are shown in Table 2. We compare the effectiveness of the reversed backdoor assuming the target class is known. We can see that previous backdoor inversion methods (NC, TopoPrior and PixelInv) all fail to produce effective backdoors in this setting. Both SmoothInv w/ diffusion and w/o diffusion find a highly effective backdoor for all cases. SmoothInv also outperforms a simplified baseline PlainAdv, suggesting that the robustification process of constructing a robust smoothed classifier is the key to the success of our approach.

We find that SmoothInv w/o diffusion outperforms w/ diffusion. We attribute this to an observation from Figure 4, where the original backdoor are more effective for SmoothInv w/o diffusion than w/ diffusion. Thus the higher the backdoor ASR is for the smoothed classifier, the more likely it is to reconstruct a more effective backdoor with SmoothInv. This verifies our hypothesis earlier, where we do not necessarily need high clean accuracies for the smoothed classifier, but what is essential is that the backdoor remains effective after the smoothing procedure.

In Figure 8, we show the ASRs of synthesized perturbations from the target class (blue) versus normal non-targeted class (red). We can see that within a reasonble range of perturbation size, the synthesized perturbations are a valid backdoor only when it is guided from the true backdoored class (Equation 6). Using this property, we find that we can successfully identify the target class for the four backdoored classifiers we consider in this work, where the synthesized patterns only have high ASRs for the correct target class. This suggests the possibility of our approach to the application of backdoor detection, we leave it as a promising extension for future work.

4.3 Qualitative Evaluation

For each example, we show the synthesized patterns with the highest ASRs among SmoothInv w/ diffusion and SmoothInv w/o diffusion.

TrojAI and HTBA

. We first show results for models with relatively large backdoors. In Figure 6, we show both pairs of clean/synthesized images for the TrojAI and HTBA backdoored classifiers. For TrojAI, synthesized images all contain a concentrated region of green pixels , matching the original polygon trigger in Figure 2. What’s more, SmoothInv synthesizes all backdoor patterns in the foreground object, which is exactly the place where the backdoor is placed during training. For HTBA, SmoothInv synthesizes patterns in the forms of small isolated color patches, e.g. red, green and blue, while these colors are all present in the original square backdoor in Figure 2. The locations of these patterns vary across images, which could be due to the random placement of the original backdoor during training.

Blind Backdoor

The results for the Blind-P and Blind-S models from blind backdoor attacks can be found in Figure 7. The synthesized images are shown in the middle. We can see that SmoothInv automatically identify the region to synthesize the backdoored patterns (in this case the top left corner), which turns out to be the exact place where the original backdoor is placed. For better comparison with the original pixel pattern and single pixel backdoors in Figure 2, we also show the zoomed in version of the $50\times 50$ top left region. Though SmoothInv does not recover the exact original backdoor, the synthesized backdoor patterns have similar visual properties to the original one, e.g., a stark contrast of white pixels and neighboring pixels. Moreover, we use these synthesized perturbations directly as additive backdoors and find that they achieve high ASRs of 99.87$\%$ and 98.35$\%$ on the Blind-P and Blind-S models respectively.

Diverse Initial Conditions

One reason for the good synthesis results in Figure 7 could be that the clean image already has a smooth background in the region of interest, i.e., top left corner, which could make the synthesis process easier. We investigate how SmoothInv is affected by the initialization of starting images. Note that for TrojAI result in Figure 6, we already show one example with rainy effects and darker background where SmoothInv still synthesizes faithful backdoor patterns. Here we analyze the Blind-P model. We select two images from ImageNet testset with non-uniform color regions (high variance in pixel values) in the top-left corner as starting images. We also use two artificial inputs: images with pure white/black pixels. We show the results of SmoothInv on these images in Figure 9. We can see that with various conditions of starting images, our approach consistently synthesizes backdoor patterns in the top left region, while the synthesized perturbations itself achieve high ASRs as well.

Additional visualization results are provided in Appendix D, where we include a direct comparison between SmoothInv w/ diffusion and w/o diffusion in Figure 13.

4.4 Mitigation of Adaptive Attacks

So far our experiments have focused on backdoor inversion on backdoored classifiers obtained from previous backdoor attacks. However, given our proposed method, someone could design new backdoor attacks to bypass our method, i.e. making it hard to extract effective backdoors with SmoothInv. The core step of SmoothInv is converting a standard backdoored classifier to a robust smoothed classifier, from which we can obtain perceptually-aligned gradients to reveal backdoor patterns. An adaptive attacker would try to circumvent SmoothInv by targeting the smoothing procedure: making the original backdoor ineffective for the smoothed classifier. Here we propose two adaptive attack attempts and show that SmoothInv is still robust in those challenging settings.

Gaussian Backdoor

One can target the SmoothInv procedure by designing a backdoor which is hardly effective on the backdoored classifier after going through the smoothing process, i.e., $\mathcal{T}(\mathcal{B}(x)+\delta)$. One immediate choice is to use a backdoor with pure Gaussian noise: a gaussian backdoor $\mathcal{B}_{g}$. With such backdoor, the backdoor information can be obfuscated after the process $\mathcal{T}(\mathcal{B}_{g}(x)+\delta)$ as $\delta$ is also gaussian noise. We construct a gaussian backdoor of size $10\times 10$, sampled from $\mathcal{N}(0,I)$ (see Figure 10 left). We use blind backdoor [2] to obtain a backdoored ImageNet classifier with this gaussian backdoor, which we call Blind-G. We are able to achieve an ASR of $100.00\%$.

On first inspection, we find that this simple gaussian backdoor does invade the smoothing procedure of SmoothInv w/ diffusion: the gaussian backdoor has an ASR of zero even for smoothed classifier constructed with noise level $\sigma=0.12$. We attribute this to the use of diffusion denoiser $\mathcal{D}$, where the Blind-G model becomes insensitive to the diffusion denoised backdoored images $\mathcal{D}(\mathcal{B}_{g}(x)+\delta)$. However, we find that the gaussian backdoor still remains highly effective for smoothed classifiers (SmoothInv w/ diffusion) constructed purely from the Blind-G model (Equation 2), despite a high drop of clean accuracy. We then apply SmoothInv w/o diffusion to the Blind-G model and achieve an average ASR of $64.84\%/91.24\%$ ($\epsilon=10/20$) for reversed backdoors from single images. We visualize some synthesized backdoor patterns in Figure 10. We can see that a dense colorful pattern emerges in the top left region via our SmoothInv process.

Training-Time Intervention

One could also make the backdoor ineffective for the smoothed classifier by modifying the training procedure. For SmoothInv, the backdoored classifier sees the processed images $\mathcal{T}(x+\delta)$ instead of the original image $x$. An adaptive attacker can intentionally make the backdoored classifier misclassify backdoored images $\mathcal{B}(x)$ while classifying $\mathcal{T}(\mathcal{B}(x)+\delta)$ correctly. To investigate if this is possible, we design a new training objective below: (we consider SmoothInv w/o diffusion due to resource limitations.)

[TABLE]

We use the pixel pattern and single pixel backdoors in Figure 2 and train classifiers with the new objective: Blind-P*/S* ($\alpha_{0},\alpha_{1},\alpha_{2}$ are automatically adjusted following [2]). We summarize the clean and backdoor accuracy of these models in Table 3. We can see that the base classifiers Blind-P*/S* have lower backdoor ASR compared to Blind-P/S, suggesting that correctly classifying $\mathcal{T}(\mathcal{B}(x)+\delta)$ affects the effectiveness of backdoor attacks in a negative way. We also study how training-time intervention affects the effectiveness of SmoothInv on Blind-P* (attack considered successful). We find that we are still able to synthesize effective backdoor perturbations with an average ASR of $88.81\%$ over 10 random starting images.

5 Conclusion and Discussion

In this paper, we have presented a method for backdoor inversion using a single clean image from the underlying data distribution. Unlike previous optimization-based approaches, our method exploits recent advances in adversarial robustness to create a smoothed version of a classifier, and then modify the image to extract the backdoor via this robust smoothed classifier. We show that SmoothInv is able to recover backdoor perturbations that are both highly successful and extremely visually similar to the true underlying backdoor. Going forward, the work suggests that many current approaches to producing backdoored classifiers can easily be “reverse engineered” to recover the underlying backdoor, which can provide a powerful mechanism to analyze the security of existing classifiers.

One major limitation of SmoothInv is that it does not generalize to more advanced backdoors. In this work, we consider patch-based backdoor in particular. However, other forms of backdoor are shown possible by previous work, e.g. image wrapping [26], adaptive imperceptible perturbation [45] and instagram filters [15]. Our method does not apply to those backdoors and we believe the likely reason is that $\ell_{2}$ based perturbations are only suitable for reversing patch-based backdoors. For advanced backdoors, we suspect that one would need to design the perturbation space of SmoothInv more carefully. For instance, we can model instagram filters with a per-pixel position-dependent transformation implemented by a neural network [19]. It would be interesting future work to extend our approach beyond patch based backdoors.

Acknowledgments.

We thank Florian Tramèr for valuable discussions during the development of this work. We thank Giulia Fanti, Aditi Raghunathan, Leslie Rice and Stephanie Rosenthal for reviewing prior draft of this work. Mingjie Sun was supported by funding from the Bosch Center for Artificial Intelligence.

Appendix A CLIP Backdoor [6]

CLIP Backdoor [6] demonstrated a backdoor attack on a multimodal contrastive model CLIP [31]. It uses a patch backdoor: a $16\times 16$ grid of black and white pixels. We train a backdoored CLIP following the steps described in [6]. We visualize a sample backdoored image in Figure 11. The backdoored CLIP model we trained has a backdoor ASR of $99.99\%$. We then cast this CLIP model as a standard image classifier in a zero-shot manner and apply SmoothInv w/o diffusion only (due to limited computation resources). We test this on 10 random clean images from ImageNet testset and the average ASRs of the reversed backdoors are $73.16\%$ for ($\epsilon=5$) and $93.51\%$ for ($\epsilon=10$). We show the reversed backdoored images in Figure 12. We can see that some patterns with high color contrast appear in the top-left part.

Appendix B Runtime and Resource Considerations

The major time bottleneck of our approach SmoothInv comes from back-propagating through the smoothed classifier $G_{b}$. SmoothInv w/o diffusion is generally fast as diffusion model is not involved. Here we mainly study the resource consumption of SmoothInv w/ diffusion. First we would like to clarify that for SmoothInv w/ diffusion, we are not using the full standard reverse diffusion process but the one-shot denoising approach proposed in [7], where we apply only one diffusion step to obtain an estimate of the denoised image. All our experiments were run on four RTX A6000 GPU machines with 48685MiB GPU memory each. In Table 4, we report the time to synthesize an image using SmoothInv with various number of noise vectors $N$. We compare with the baseline “Standard” where we back-propagate through the standard base classifier (ResNet-18). We can see that the time spent scales linearly with the number of noisy vectors. In our experiments, we find that $N=10$ noise vectors are usually enough for a stable synthesis result. In this case, SmoothInv takes roughly around 5 mins to synthesize one backdoored image in one GPU. It would be interesting future work to investigate methods to speedup our synthesis process when a diffusion denoiser is used.

Appendix C Pseudo-code

We provide a PyTorch-style pseudo-code for our approach SmoothInv in Algorithm 1, where we apply projected gradient descent [24] to synthesize backdoored patterns given a single image. We use a pre-trained diffusion model to build a robust smoothed classifier, following [7].

Appendix D Additional Visualization Results

In Figure 16, we show some backdoored images with the true backdoors listed in Table 2. Notice that for Blind-P and Blind-S, the backdoors are placed in the top left region of the images (the injected backdoor may be hard to identify unless zooming in the specific part).

We provide a comparison of SmoothInv w/ diffusion and w/o diffusion in Figure 13. We can see that for TrojAI, SmoothInv w/o diffusion tends to generate more regions of interest on the background while the synthesized patterns appear more often in the foreground for SmoothInv w/ diffusion. For HTBA, SmoothInv w/o diffusion tends to have vague artifacts while the backdoor patterns are more distinctive for SmoothInv w/ diffusion. This suggests that while SmoothInv w/o diffusion may generate more effective backdoors, but using a diffusion denoiser may lead to better visualization results.

We include more visualization results ($\epsilon=10$) on the Blind-P and Blind-S models in Figure 14 and Figure 15, where we show the synthesized images under various noise level $\sigma\in\{0.25,0.50,1.00\}$. We find no distinction between w/ diffusion and w/o diffusion visually so here we show the results of SmoothInv w/o diffusion for these two models. We can see that in general, smoothed classifiers constructed with larger noise levels tend to give better visualization results.

Appendix E Sanity Check

Following the initial sanity check result in Figure 4, we report the results on all four backdoored classifiers in Table 5. We show both the clean accuracy and backdoor ASR of the smoothed classifiers *w/ * and w/o diffusion. We can see that using a diffusion denoiser can significantly improve the clean accuracy of the resulting smoothed classifiers for all four backdoored classifiers. For backdoor ASR, we can see that the backdoor remains effective for smoothed classifiers both w/ and w/o diffusion for some values of noise level $\sigma$. Note that for TrojAI, the results should have large variance as only five clean images are provided by the data publisher.