Deep Adversarially-Enhanced k-Nearest Neighbors

TL;DR

This paper introduces DAEkNN, a method that enhances the robustness of deep neural networks against adversarial attacks by combining adversarial training with a weighted k-nearest neighbors approach, improving robustness-accuracy trade-offs.

Contribution

The paper proposes DAEkNN, a novel approach that integrates adversarial training with a weighted k-nearest neighbors classifier to improve robustness in deep neural networks.

Findings

DAEkNN outperforms DkNN in robustness on MNIST and CIFAR-10.

DAEkNN mitigates the robustness-accuracy trade-off in deep layers.

Empirical results show improved robustness and trade-off balance.

Abstract

Recent works have theoretically and empirically shown that deep neural networks (DNNs) have an inherent vulnerability to small perturbations. Applying the Deep k-Nearest Neighbors (DkNN) classifier, we observe a dramatically increasing robustness-accuracy trade-off as the layer goes deeper. In this work, we propose a Deep Adversarially-Enhanced k-Nearest Neighbors (DAEkNN) method which achieves higher robustness than DkNN and mitigates the robustness-accuracy trade-off in deep layers through two key elements. First, DAEkNN is based on an adversarially trained model. Second, DAEkNN makes predictions by leveraging a weighted combination of benign and adversarial training data. Empirically, we find that DAEkNN improves both the robustness and the robustness-accuracy trade-off on MNIST and CIFAR-10 datasets.