DLL: A Blazing Fast Deep Neural Network Library

TL;DR

DLL is a new deep learning library optimized for speed, supporting various neural network models and demonstrating significant performance improvements over existing frameworks on CPU and GPU without sacrificing accuracy.

Contribution

The paper introduces DLL, a deep learning library with novel software engineering strategies that significantly accelerate training and inference across multiple neural network types.

Findings

DLL is faster than five popular frameworks on CPU and GPU.

DLL achieves comparable classification accuracy to other frameworks.

The proposed strategies are largely independent of specific neural network algorithms.

Abstract

Deep Learning Library (DLL) is a new library for machine learning with deep neural networks that focuses on speed. It supports feed-forward neural networks such as fully-connected Artificial Neural Networks (ANNs) and Convolutional Neural Networks (CNNs). It also has very comprehensive support for Restricted Boltzmann Machines (RBMs) and Convolutional RBMs. Our main motivation for this work was to propose and evaluate novel software engineering strategies with potential to accelerate runtime for training and inference. Such strategies are mostly independent of the underlying deep learning algorithms. On three different datasets and for four different neural network models, we compared DLL to five popular deep learning frameworks. Experimentally, it is shown that the proposed framework is systematically and significantly faster on CPU and GPU. In terms of classification performance, similar accuracies as the other frameworks are reported.