A Manifold View of Adversarial Risk

TL;DR

This paper introduces a manifold-based framework for analyzing adversarial risk in machine learning, distinguishing between normal and in-manifold perturbations, and provides bounds and empirical evidence for these risks.

Contribution

It proposes a novel manifold perspective on adversarial risk, defining new risk types and establishing bounds relating them to the classic adversarial risk.

Findings

Normal and in-manifold risks can bound the classic adversarial risk.

Standard adversarial risk can be nonzero even if both risks are zero.

Focusing on normal adversarial risk may improve classifier robustness.

Abstract

The adversarial risk of a machine learning model has been widely studied. Most previous works assume that the data lies in the whole ambient space. We propose to take a new angle and take the manifold assumption into consideration. Assuming data lies in a manifold, we investigate two new types of adversarial risk, the normal adversarial risk due to perturbation along normal direction, and the in-manifold adversarial risk due to perturbation within the manifold. We prove that the classic adversarial risk can be bounded from both sides using the normal and in-manifold adversarial risks. We also show with a surprisingly pessimistic case that the standard adversarial risk can be nonzero even when both normal and in-manifold risks are zero. We finalize the paper with empirical studies supporting our theoretical results. Our results suggest the possibility of improving the robustness of a classifier by only focusing on the normal adversarial risk.