A Le Cam Type Bound for Adversarial Learning and Applications

TL;DR

This paper establishes fundamental information-theoretic limits for adversarial learning, providing a Le Cam type bound that applies broadly without assuming specific attack strategies, and demonstrates its relevance through applications to key learning problems.

Contribution

It introduces a general Le Cam type bound for adversarial learning that does not depend on specific attack models, advancing understanding of fundamental robustness limits.

Findings

Derived a universal bound for adversarial learning performance

Applied bounds to canonical learning problems with common attack types

Highlights limitations of existing defense strategies

Abstract

Robustness of machine learning methods is essential for modern practical applications. Given the arms race between attack and defense methods, one may be curious regarding the fundamental limits of any defense mechanism. In this work, we focus on the problem of learning from noise-injected data, where the existing literature falls short by either assuming a specific attack method or by over-specifying the learning problem. We shed light on the information-theoretic limits of adversarial learning without assuming a particular learning process or attacker. Finally, we apply our general bounds to a canonical set of non-trivial learning problems and provide examples of common types of attacks.