# Neural ODEs as the Deep Limit of ResNets with constant weights

## TL;DR

This paper provides a theoretical foundation showing that ResNets with shared weights converge to Neural ODEs in the deep limit, connecting discrete deep networks with continuous models.

## Contribution

It proves that stochastic gradient descent on shared-weight ResNets converges to that on Neural ODEs, establishing a formal link between the two.

## Key findings

- ResNets with constant weights approximate Neural ODEs in the deep limit
- The convergence of loss functions is established
- Fokker-Planck equations are used for the proof

## Abstract

In this paper we prove that, in the deep limit, the stochastic gradient descent on a ResNet type deep neural network, where each layer shares the same weight matrix, converges to the stochastic gradient descent for a Neural ODE and that the corresponding value/loss functions converge. Our result gives, in the context of minimization by stochastic gradient descent, a theoretical foundation for considering Neural ODEs as the deep limit of ResNets. Our proof is based on certain decay estimates for associated Fokker-Planck equations.