Deep Neural Networks

TL;DR

Deep Neural Networks are powerful but lack a comprehensive mathematical framework, stability guarantees, and interpretability, which are crucial for their reliable deployment in societal applications.

Contribution

This paper highlights key theoretical challenges in DNNs, including interpretability, stability, generalization, and unlabeled data utilization, aiming to guide future research.

Findings

Identifies fundamental gaps in DNN theory and practice.

Emphasizes the importance of interpretability and stability.

Calls for developing a unified mathematical framework.

Abstract

Deep Neural Networks (DNNs) are universal function approximators providing state-of- the-art solutions on wide range of applications. Common perceptual tasks such as speech recognition, image classification, and object tracking are now commonly tackled via DNNs. Some fundamental problems remain: (1) the lack of a mathematical framework providing an explicit and interpretable input-output formula for any topology, (2) quantification of DNNs stability regarding adversarial examples (i.e. modified inputs fooling DNN predictions whilst undetectable to humans), (3) absence of generalization guarantees and controllable behaviors for ambiguous patterns, (4) leverage unlabeled data to apply DNNs to domains where expert labeling is scarce as in the medical field. Answering those points would provide theoretical perspectives for further developments based on a common ground. Furthermore, DNNs are now deployed in tremendous societal applications, pushing the need to fill this theoretical gap to ensure control, reliability, and interpretability.