On the Geometry of Deep Learning

TL;DR

This paper explores the geometric properties of deep neural networks, focusing on how they approximate functions using affine splines and tessellate input space, offering insights into their inner workings.

Contribution

It reviews a decade of research linking deep networks to affine splines and their geometric tessellations, providing a new perspective on understanding deep learning models.

Findings

Deep networks can be viewed as affine splines tessellating input space.

Geometric analysis offers insights into network function approximation.

Understanding these properties can guide network design improvements.

Abstract

In this paper, we overview one promising avenue of progress at the mathematical foundation of deep learning: the connection between deep networks and function approximation by affine splines (continuous piecewise linear functions in multiple dimensions). In particular, we will overview work over the past decade on understanding certain geometrical properties of a deep network's affine spline mapping, in particular how it tessellates its input space. As we will see, the affine spline connection and geometrical viewpoint provide a powerful portal through which to view, analyze, and improve the inner workings of a deep network.