Decision boundaries and convex hulls in the feature space that deep learning functions learn from images

TL;DR

This paper investigates the geometric properties of decision boundaries and convex hulls in the feature space of deep neural networks, providing insights into model behavior, adversarial vulnerabilities, and classification ambiguities.

Contribution

It introduces methods to analyze the feature space of deep models, revealing the geometric structure of decision boundaries and their implications for image classification.

Findings

Decision boundaries in feature space differ significantly from pixel space.

Geometric analysis offers insights into adversarial vulnerabilities.

Regions with guaranteed classifications are identified in feature space.

Abstract

The success of deep neural networks in image classification and learning can be partly attributed to the features they extract from images. It is often speculated about the properties of a low-dimensional manifold that models extract and learn from images. However, there is not sufficient understanding about this low-dimensional space based on theory or empirical evidence. For image classification models, their last hidden layer is the one where images of each class is separated from other classes and it also has the least number of features. Here, we develop methods and formulations to study that feature space for any model. We study the partitioning of the domain in feature space, identify regions guaranteed to have certain classifications, and investigate its implications for the pixel space. We observe that geometric arrangements of decision boundaries in feature space is significantly different compared to pixel space, providing insights about adversarial vulnerabilities, image morphing, extrapolation, ambiguity in classification, and the mathematical understanding of image classification models.