Limitations of Deep Neural Networks: a discussion of G. Marcus' critical appraisal of deep learning

TL;DR

This paper discusses the limitations of deep neural networks, critiques Marcus' appraisal, and aims to clarify misconceptions to guide future research in deep learning.

Contribution

It provides a critical analysis of deep learning's weaknesses and suggests directions for addressing these issues and misconceptions.

Findings

Deep neural networks have achieved widespread success across various fields.

There are significant limitations and pitfalls in deep learning that need addressing.

Misconceptions about deep learning may hinder scientific progress.

Abstract

Deep neural networks have triggered a revolution in artificial intelligence, having been applied with great results in medical imaging, semi-autonomous vehicles, ecommerce, genetics research, speech recognition, particle physics, experimental art, economic forecasting, environmental science, industrial manufacturing, and a wide variety of applications in nearly every field. This sudden success, though, may have intoxicated the research community and blinded them to the potential pitfalls of assigning deep learning a higher status than warranted. Also, research directed at alleviating the weaknesses of deep learning may seem less attractive to scientists and engineers, who focus on the low-hanging fruit of finding more and more applications for deep learning models, thus letting short-term benefits hamper long-term scientific progress. Gary Marcus wrote a paper entitled Deep Learning: A Critical Appraisal, and here we discuss Marcus' core ideas, as well as attempt a general assessment of the subject. This study examines some of the limitations of deep neural networks, with the intention of pointing towards potential paths for future research, and of clearing up some metaphysical misconceptions, held by numerous researchers, that may misdirect them.