OTAD: An Optimal Transport-Induced Robust Model for Agnostic Adversarial Attack

TL;DR

OTAD introduces a novel robust deep learning model that combines optimal transport theory with Lipschitz regularization, enhancing resistance to adversarial attacks while maintaining high data fidelity across various architectures.

Contribution

The paper proposes a new two-step OTAD model that integrates optimal transport regularization and convex integration to improve adversarial robustness and model expressiveness.

Findings

OTAD outperforms existing robust models on multiple datasets.

The method is compatible with ResNet and Transformer architectures.

OTAD effectively balances robustness and accuracy.

Abstract

Deep neural networks (DNNs) are vulnerable to small adversarial perturbations of the inputs, posing a significant challenge to their reliability and robustness. Empirical methods such as adversarial training can defend against particular attacks but remain vulnerable to more powerful attacks. Alternatively, Lipschitz networks provide certified robustness to unseen perturbations but lack sufficient expressive power. To harness the advantages of both approaches, we design a novel two-step Optimal Transport induced Adversarial Defense (OTAD) model that can fit the training data accurately while preserving the local Lipschitz continuity. First, we train a DNN with a regularizer derived from optimal transport theory, yielding a discrete optimal transport map linking data to its features. By leveraging the map's inherent regularity, we interpolate the map by solving the convex integration problem (CIP) to guarantee the local Lipschitz property. OTAD is extensible to diverse architectures of ResNet and Transformer, making it suitable for complex data. For efficient computation, the CIP can be solved through training neural networks. OTAD opens a novel avenue for developing reliable and secure deep learning systems through the regularity of optimal transport maps. Empirical results demonstrate that OTAD can outperform other robust models on diverse datasets.