Mathematics of Deep Learning
Rene Vidal, Joan Bruna, Raja Giryes, Stefano Soatto
TL;DR
This paper reviews recent mathematical work explaining why deep learning architectures perform well, focusing on properties like optimality, stability, and invariance of learned representations.
Contribution
It provides a comprehensive overview of mathematical justifications for key properties of deep networks, enhancing understanding of their success.
Findings
Deep networks can achieve global optimality under certain conditions
Mathematical frameworks explain geometric stability of deep representations
Invariance properties of learned features are supported by recent theoretical work
Abstract
Recently there has been a dramatic increase in the performance of recognition systems due to the introduction of deep architectures for representation learning and classification. However, the mathematical reasons for this success remain elusive. This tutorial will review recent work that aims to provide a mathematical justification for several properties of deep networks, such as global optimality, geometric stability, and invariance of the learned representations.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSparse and Compressive Sensing Techniques · Neural Networks and Applications · Stochastic Gradient Optimization Techniques