Meet You Halfway: Explaining Deep Learning Mysteries

TL;DR

This paper introduces a new conceptual framework to better understand deep neural networks, explaining their generalization abilities and the transferability of adversarial examples through formal analysis and experiments.

Contribution

It presents a novel framework with formal descriptions that elucidates key mysteries of deep learning, including generalization and adversarial transferability.

Findings

Framework explains why neural networks generalize well.

Framework clarifies why adversarial examples transfer across models.

Experimental results support the proposed theoretical explanations.

Abstract

Deep neural networks perform exceptionally well on various learning tasks with state-of-the-art results. While these models are highly expressive and achieve impressively accurate solutions with excellent generalization abilities, they are susceptible to minor perturbations. Samples that suffer such perturbations are known as "adversarial examples". Even though deep learning is an extensively researched field, many questions about the nature of deep learning models remain unanswered. In this paper, we introduce a new conceptual framework attached with a formal description that aims to shed light on the network's behavior and interpret the behind-the-scenes of the learning process. Our framework provides an explanation for inherent questions concerning deep learning. Particularly, we clarify: (1) Why do neural networks acquire generalization abilities? (2) Why do adversarial examples transfer between different models?. We provide a comprehensive set of experiments that support this new framework, as well as its underlying theory.