MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems

TL;DR

MXNet is a versatile, efficient machine learning library designed for heterogeneous systems, combining symbolic and imperative programming to facilitate deep neural network development across diverse hardware.

Contribution

The paper introduces MXNet's unified API and system architecture, enabling flexible deep learning development on various hardware platforms.

Findings

Demonstrated promising large-scale DNN training results

Achieved computation and memory efficiency across systems

Supported multi-language and heterogeneous hardware environments

Abstract

MXNet is a multi-language machine learning (ML) library to ease the development of ML algorithms, especially for deep neural networks. Embedded in the host language, it blends declarative symbolic expression with imperative tensor computation. It offers auto differentiation to derive gradients. MXNet is computation and memory efficient and runs on various heterogeneous systems, ranging from mobile devices to distributed GPU clusters. This paper describes both the API design and the system implementation of MXNet, and explains how embedding of both symbolic expression and tensor operation is handled in a unified fashion. Our preliminary experiments reveal promising results on large scale deep neural network applications using multiple GPU machines.