Understanding the Decision Boundary of Deep Neural Networks: An Empirical Study

TL;DR

This paper empirically investigates how the decision boundary of deep neural networks evolves during training, revealing that it moves closer to natural images over time, and explores how adversarial training can mitigate this effect.

Contribution

It provides the first detailed empirical analysis of decision boundary dynamics during training and highlights the potential of adversarial training to improve robustness.

Findings

Decision boundary moves closer to natural images over training.

Adversarial training can prevent the decision boundary from converging too closely.

The phenomenon persists even in late training epochs.

Abstract

Despite achieving remarkable performance on many image classification tasks, state-of-the-art machine learning (ML) classifiers remain vulnerable to small input perturbations. Especially, the existence of adversarial examples raises concerns about the deployment of ML models in safety- and security-critical environments, like autonomous driving and disease detection. Over the last few years, numerous defense methods have been published with the goal of improving adversarial as well as corruption robustness. However, the proposed measures succeeded only to a very limited extent. This limited progress is partly due to the lack of understanding of the decision boundary and decision regions of deep neural networks. Therefore, we study the minimum distance of data points to the decision boundary and how this margin evolves over the training of a deep neural network. By conducting experiments on MNIST, FASHION-MNIST, and CIFAR-10, we observe that the decision boundary moves closer to natural images over training. This phenomenon even remains intact in the late epochs of training, where the classifier already obtains low training and test error rates. On the other hand, adversarial training appears to have the potential to prevent this undesired convergence of the decision boundary.