An Analysis of Robustness of Non-Lipschitz Networks

TL;DR

This paper investigates the robustness of non-Lipschitz neural networks against adversarial attacks, proposing a theoretical framework that explains their vulnerabilities and potential defenses through abstention and data-driven parameter tuning.

Contribution

It introduces a novel attack model based on low-dimensional subspace perturbations, analyzes the limitations of classifiers under this model, and offers robustness guarantees for abstention strategies.

Findings

Adversaries can be powerful within the proposed model, defeating classifiers without abstention.

Allowing abstention enables classifiers to maintain high accuracy against adversaries.

Empirical results show high robust accuracy with low abstention rates in contrastive learning.

Abstract

Despite significant advances, deep networks remain highly susceptible to adversarial attack. One fundamental challenge is that small input perturbations can often produce large movements in the network's final-layer feature space. In this paper, we define an attack model that abstracts this challenge, to help understand its intrinsic properties. In our model, the adversary may move data an arbitrary distance in feature space but only in random low-dimensional subspaces. We prove such adversaries can be quite powerful: defeating any algorithm that must classify any input it is given. However, by allowing the algorithm to abstain on unusual inputs, we show such adversaries can be overcome when classes are reasonably well-separated in feature space. We further provide strong theoretical guarantees for setting algorithm parameters to optimize over accuracy-abstention trade-offs using data-driven methods. Our results provide new robustness guarantees for nearest-neighbor style algorithms, and also have application to contrastive learning, where we empirically demonstrate the ability of such algorithms to obtain high robust accuracy with low abstention rates. Our model is also motivated by strategic classification, where entities being classified aim to manipulate their observable features to produce a preferred classification, and we provide new insights into that area as well.