Deep Residual Learning and PDEs on Manifold

TL;DR

This paper links deep residual networks to control problems of PDEs on manifolds, proposing models based on transport and Hamilton-Jacobi equations, and discusses discretization on point clouds.

Contribution

It introduces a novel perspective by formulating ResNet as a control problem of PDEs on manifolds and proposes new models based on transport and Hamilton-Jacobi equations.

Findings

ResNet can be interpreted as solving a transport PDE along characteristics.

New models based on transport and Hamilton-Jacobi equations are proposed.

Discretization methods for PDEs on point clouds are discussed.

Abstract

In this paper, we formulate the deep residual network (ResNet) as a control problem of transport equation. In ResNet, the transport equation is solved along the characteristics. Based on this observation, deep neural network is closely related to the control problem of PDEs on manifold. We propose several models based on transport equation and Hamilton-Jacobi equation. The discretization of these PDEs on point cloud is also discussed.