Some Geometrical and Topological Properties of DNNs' Decision Boundaries

TL;DR

This paper uses differential geometry to analyze the geometrical and topological properties of DNN decision boundaries, providing insights for model design and regularization to improve robustness.

Contribution

It introduces a theoretical framework linking DNN parameters to decision boundary geometry and topology, including curvature and Euler characteristic calculations.

Findings

Derived formulas for decision boundary curvature based on network parameters.

Provided sufficient conditions for flat or developable decision boundaries.

Verified Euler characteristic computation method through experiments.

Abstract

Geometry and topology of decision regions are closely related with classification performance and robustness against adversarial attacks. In this paper, we use differential geometry to theoretically explore the geometrical and topological properties of decision regions produced by deep neural networks (DNNs). The goal is to obtain some geometrical and topological properties of decision boundaries for given DNN models, and provide some principled guidance to design and regularization of DNNs. First, we present the curvatures of decision boundaries in terms of network parameters, and give sufficient conditions on network parameters for producing flat or developable decision boundaries. Based on the Gauss-Bonnet-Chern theorem in differential geometry, we then propose a method to compute the Euler characteristics of compact decision boundaries, and verify it with experiments.