Deep Learning At Scale and At Ease

TL;DR

This paper introduces SINGA, a distributed deep learning platform designed to be user-friendly and scalable, capable of handling large models and datasets efficiently across CPU and GPU systems.

Contribution

The paper presents SINGA, a novel deep learning platform with an intuitive programming model and optimized distributed architecture for improved usability and scalability.

Findings

SINGA outperforms many state-of-the-art systems in scalability and efficiency.

The platform is effective for real-life multimedia applications.

SINGA supports both CPU and GPU environments.

Abstract

Recently, deep learning techniques have enjoyed success in various multimedia applications, such as image classification and multi-modal data analysis. Large deep learning models are developed for learning rich representations of complex data. There are two challenges to overcome before deep learning can be widely adopted in multimedia and other applications. One is usability, namely the implementation of different models and training algorithms must be done by non-experts without much effort especially when the model is large and complex. The other is scalability, that is the deep learning system must be able to provision for a huge demand of computing resources for training large models with massive datasets. To address these two challenges, in this paper, we design a distributed deep learning platform called SINGA which has an intuitive programming model based on the common layer abstraction of deep learning models. Good scalability is achieved through flexible distributed training architecture and specific optimization techniques. SINGA runs on GPUs as well as on CPUs, and we show that it outperforms many other state-of-the-art deep learning systems. Our experience with developing and training deep learning models for real-life multimedia applications in SINGA shows that the platform is both usable and scalable.