Sparse Deep Neural Network Graph Challenge

TL;DR

The paper introduces a scalable Sparse Deep Neural Network (DNN) inference challenge based on well-defined computations and real datasets, aimed at benchmarking and advancing sparse AI systems across diverse hardware.

Contribution

It presents a new scalable sparse DNN inference challenge with standardized specifications, enabling performance measurement and comparison of current and future AI hardware systems.

Findings

Reference implementations demonstrate measurable serial and parallel performance.

The challenge is scalable in problem size and hardware.

It provides a rigorous framework for benchmarking sparse DNN inference.

Abstract

The MIT/IEEE/Amazon GraphChallenge.org encourages community approaches to developing new solutions for analyzing graphs and sparse data. Sparse AI analytics present unique scalability difficulties. The proposed Sparse Deep Neural Network (DNN) Challenge draws upon prior challenges from machine learning, high performance computing, and visual analytics to create a challenge that is reflective of emerging sparse AI systems. The Sparse DNN Challenge is based on a mathematically well-defined DNN inference computation and can be implemented in any programming environment. Sparse DNN inference is amenable to both vertex-centric implementations and array-based implementations (e.g., using the GraphBLAS.org standard). The computations are simple enough that performance predictions can be made based on simple computing hardware models. The input data sets are derived from the MNIST handwritten letters. The surrounding I/O and verification provide the context for each sparse DNN inference that allows rigorous definition of both the input and the output. Furthermore, since the proposed sparse DNN challenge is scalable in both problem size and hardware, it can be used to measure and quantitatively compare a wide range of present day and future systems. Reference implementations have been implemented and their serial and parallel performance have been measured. Specifications, data, and software are publicly available at GraphChallenge.org