Deep Learning: Computational Aspects

TL;DR

This paper reviews the computational challenges of deep learning, emphasizing the importance of efficient linear algebra, stochastic gradient descent, and batch sampling in training deep neural networks on large datasets.

Contribution

It provides a comprehensive overview of the computational techniques and considerations essential for effective deep learning model training.

Findings

Efficient linear algebra libraries are crucial for deep learning training.

Stochastic gradient descent and batch sampling enable learning from large datasets.

Computational aspects significantly impact deep learning performance and scalability.

Abstract

In this article we review computational aspects of Deep Learning (DL). Deep learning uses network architectures consisting of hierarchical layers of latent variables to construct predictors for high-dimensional input-output models. Training a deep learning architecture is computationally intensive, and efficient linear algebra libraries is the key for training and inference. Stochastic gradient descent (SGD) optimization and batch sampling are used to learn from massive data sets.