Reinforcing Adversarial Robustness using Model Confidence Induced by Adversarial Training

TL;DR

This paper proposes a framework called HCNN that enhances adversarial robustness by leveraging confidence measures from adversarial training and nearest neighbor search, supported by empirical results.

Contribution

It introduces HCNN, a novel method combining confidence information and nearest neighbor search to improve adversarial robustness of trained models.

Findings

Confidence can distinguish correct from incorrect predictions near data points.

HCNN improves robustness against adversarial attacks in experiments.

Existing adversarial training has limitations that HCNN aims to address.

Abstract

In this paper we study leveraging confidence information induced by adversarial training to reinforce adversarial robustness of a given adversarially trained model. A natural measure of confidence is $\|F({\bf x})\|_\infty$ (i.e. how confident $F$ is about its prediction?). We start by analyzing an adversarial training formulation proposed by Madry et al.. We demonstrate that, under a variety of instantiations, an only somewhat good solution to their objective induces confidence to be a discriminator, which can distinguish between right and wrong model predictions in a neighborhood of a point sampled from the underlying distribution. Based on this, we propose Highly Confident Near Neighbor (${\tt HCNN}$), a framework that combines confidence information and nearest neighbor search, to reinforce adversarial robustness of a base model. We give algorithms in this framework and perform a detailed empirical study. We report encouraging experimental results that support our analysis, and also discuss problems we observed with existing adversarial training.