Challenges in Migrating Imperative Deep Learning Programs to Graph Execution: An Empirical Study

TL;DR

This empirical study investigates the challenges and bugs encountered when migrating imperative deep learning programs to graph execution, revealing issues like API misuse and performance degradation, and offers practical recommendations.

Contribution

The paper provides a comprehensive data-driven analysis of real-world challenges in hybrid imperative and graph-based deep learning frameworks, highlighting common pitfalls and best practices.

Findings

Hybrid approaches are prone to API misuse.

Performance degradation can occur with hybridization.

Limited applicability due to execution mode incompatibility.

Abstract

Efficiency is essential to support responsiveness w.r.t. ever-growing datasets, especially for Deep Learning (DL) systems. DL frameworks have traditionally embraced deferred execution-style DL code that supports symbolic, graph-based Deep Neural Network (DNN) computation. While scalable, such development tends to produce DL code that is error-prone, non-intuitive, and difficult to debug. Consequently, more natural, less error-prone imperative DL frameworks encouraging eager execution have emerged but at the expense of run-time performance. While hybrid approaches aim for the "best of both worlds," the challenges in applying them in the real world are largely unknown. We conduct a data-driven analysis of challenges -- and resultant bugs -- involved in writing reliable yet performant imperative DL code by studying 250 open-source projects, consisting of 19.7 MLOC, along with 470 and 446 manually examined code patches and bug reports, respectively. The results indicate that hybridization: (i) is prone to API misuse, (ii) can result in performance degradation -- the opposite of its intention, and (iii) has limited application due to execution mode incompatibility. We put forth several recommendations, best practices, and anti-patterns for effectively hybridizing imperative DL code, potentially benefiting DL practitioners, API designers, tool developers, and educators.