Closing the Gap: Achieving Better Accuracy-Robustness Tradeoffs against Query-Based Attacks

TL;DR

This paper introduces a test-time method to improve the robustness-accuracy tradeoff against query-based attacks by activating defenses only on low-confidence inputs, without retraining models.

Contribution

It presents a training-free, theoretically supported approach that enhances existing defenses by selectively applying them during testing to balance robustness and accuracy.

Findings

Improves robustness-accuracy tradeoff on CIFAR-10, CIFAR-100, ImageNet

Enhances existing defenses without retraining models

Effective against query-based attacks in experiments

Abstract

Although promising, existing defenses against query-based attacks share a common limitation: they offer increased robustness against attacks at the price of a considerable accuracy drop on clean samples. In this work, we show how to efficiently establish, at test-time, a solid tradeoff between robustness and accuracy when mitigating query-based attacks. Given that these attacks necessarily explore low-confidence regions, our insight is that activating dedicated defenses, such as random noise defense and random image transformations, only for low-confidence inputs is sufficient to prevent them. Our approach is independent of training and supported by theory. We verify the effectiveness of our approach for various existing defenses by conducting extensive experiments on CIFAR-10, CIFAR-100, and ImageNet. Our results confirm that our proposal can indeed enhance these defenses by providing better tradeoffs between robustness and accuracy when compared to state-of-the-art approaches while being completely training-free.