Control on the Manifolds of Mappings with a View to the Deep Learning

TL;DR

This paper explores the use of control systems on manifolds to model and analyze deep neural networks, viewing them as continuous-time control systems with a continuum of layers, offering a new perspective on neural network approximation.

Contribution

It introduces a novel framework that treats neural networks as control systems on manifolds, connecting deep learning with control theory and differential geometry.

Findings

Neural networks can be modeled as nonlinear control systems on manifolds.

The continuum limit of layers corresponds to a control system with a continuum of parameters.

This approach provides new insights into the approximation capabilities of neural networks.

Abstract

Deep learning of the Artificial Neural Networks (ANN) can be treated as a particular class of interpolation problems. The goal is to find a neural network whose input-output map approximates well the desired map on a finite or an infinite training set. Our idea consists of taking as an approximant the input-output map, which arises from a nonlinear continuous-time control system. In the limit such control system can be seen as a network with a continuum of layers, each one labelled by the time variable. The values of the controls at each instant of time are the parameters of the layer.