Improving Local Effectiveness for Global robust training

TL;DR

This paper introduces a novel robust training algorithm that effectively utilizes weak adversaries by focusing on local input regions, achieving high robustness with reduced training time across multiple datasets.

Contribution

The proposed method enhances robustness by leveraging local adversarial regions, reducing reliance on strong adversaries, and matching or surpassing state-of-the-art results with less computational cost.

Findings

Achieves similar robustness to state-of-the-art methods with weak adversaries on MNIST, CIFAR-10, CIFAR-100.

Reduces overall training time compared to existing robust training approaches.

Outperforms current methods on CIFAR-10 and CIFAR-100 when trained with strong adversaries.

Abstract

Despite its popularity, deep neural networks are easily fooled. To alleviate this deficiency, researchers are actively developing new training strategies, which encourage models that are robust to small input perturbations. Several successful robust training methods have been proposed. However, many of them rely on strong adversaries, which can be prohibitively expensive to generate when the input dimension is high and the model structure is complicated. We adopt a new perspective on robustness and propose a novel training algorithm that allows a more effective use of adversaries. Our method improves the model robustness at each local ball centered around an adversary and then, by combining these local balls through a global term, achieves overall robustness. We demonstrate that, by maximizing the use of adversaries via focusing on local balls, we achieve high robust accuracy with weak adversaries. Specifically, our method reaches a similar robust accuracy level to the state of the art approaches trained on strong adversaries on MNIST, CIFAR-10 and CIFAR-100. As a result, the overall training time is reduced. Furthermore, when trained with strong adversaries, our method matches with the current state of the art on MNIST and outperforms them on CIFAR-10 and CIFAR-100.