Why & When Deep Learning Works: Looking Inside Deep Learnings

TL;DR

This paper explores the underlying reasons and conditions for the effectiveness of deep learning, analyzing network expressiveness, limitations, and interpretability to provide insights into their functioning.

Contribution

It offers a comprehensive analysis of why and when deep networks succeed or fail, including geometric impacts and interpretability aspects.

Findings

Deep networks' success depends on specific geometric properties.

Limitations of deep learning are linked to expressiveness constraints.

Interpretability techniques help understand deep network decisions.

Abstract

The Intel Collaborative Research Institute for Computational Intelligence (ICRI-CI) has been heavily supporting Machine Learning and Deep Learning research from its foundation in 2012. We have asked six leading ICRI-CI Deep Learning researchers to address the challenge of "Why & When Deep Learning works", with the goal of looking inside Deep Learning, providing insights on how deep networks function, and uncovering key observations on their expressiveness, limitations, and potential. The output of this challenge resulted in five papers that address different facets of deep learning. These different facets include a high-level understating of why and when deep networks work (and do not work), the impact of geometry on the expressiveness of deep networks, and making deep networks interpretable.