Mathematical Challenges in Deep Learning

TL;DR

This paper discusses the mathematical challenges in deep learning, including training, inference, and optimization, highlighting the need for formal analysis to advance the field.

Contribution

It provides a subjective overview of key mathematical problems in deep learning, aiming to bridge gaps between AI and theoretical sciences.

Findings

Identifies core mathematical challenges in deep learning

Highlights the importance of formalism for future research

Emphasizes long-term benefits for the tech industry

Abstract

Deep models are dominating the artificial intelligence (AI) industry since the ImageNet challenge in 2012. The size of deep models is increasing ever since, which brings new challenges to this field with applications in cell phones, personal computers, autonomous cars, and wireless base stations. Here we list a set of problems, ranging from training, inference, generalization bound, and optimization with some formalism to communicate these challenges with mathematicians, statisticians, and theoretical computer scientists. This is a subjective view of the research questions in deep learning that benefits the tech industry in long run.