Skyline: Interactive In-Editor Computational Performance Profiling for Deep Neural Network Training

TL;DR

Skyline is an interactive in-editor tool that helps deep learning developers debug and optimize DNN training performance through visualization, prediction, and direct manipulation of training parameters.

Contribution

It introduces a novel in-editor profiling and debugging tool that leverages DNN training properties for interactive performance analysis and parameter manipulation.

Findings

Participants found Skyline useful and easy to use.

Skyline enables real-time performance predictions and visualizations.

The tool facilitates direct code manipulation via visual interfaces.

Abstract

Training a state-of-the-art deep neural network (DNN) is a computationally-expensive and time-consuming process, which incentivizes deep learning developers to debug their DNNs for computational performance. However, effectively performing this debugging requires intimate knowledge about the underlying software and hardware systems---something that the typical deep learning developer may not have. To help bridge this gap, we present Skyline: a new interactive tool for DNN training that supports in-editor computational performance profiling, visualization, and debugging. Skyline's key contribution is that it leverages special computational properties of DNN training to provide (i) interactive performance predictions and visualizations, and (ii) directly manipulatable visualizations that, when dragged, mutate the batch size in the code. As an in-editor tool, Skyline allows users to leverage these diagnostic features to debug the performance of their DNNs during development. An exploratory qualitative user study of Skyline produced promising results; all the participants found Skyline to be useful and easy to use.