The Grand Illusion: The Myth of Software Portability and Implications for ML Progress

TL;DR

This paper investigates the portability of popular ML frameworks across hardware types, revealing significant functional and performance losses that hinder innovation and suggest hardware specialization limits progress.

Contribution

It provides the first large-scale quantification of ML framework portability issues across hardware, highlighting the costs of hardware-software co-evolution.

Findings

Frameworks can lose over 40% of functions when ported.

Performance slowdown can be extreme, making use impractical.

Specialization of hardware impedes ML research and innovation.

Abstract

Pushing the boundaries of machine learning often requires exploring different hardware and software combinations. However, the freedom to experiment across different tooling stacks can be at odds with the drive for efficiency, which has produced increasingly specialized AI hardware and incentivized consolidation around a narrow set of ML frameworks. Exploratory research can be restricted if software and hardware are co-evolving, making it even harder to stray away from mainstream ideas that work well with popular tooling stacks. While this friction increasingly impacts the rate of innovation in machine learning, to our knowledge the lack of portability in tooling has not been quantified. In this work, we ask: How portable are popular ML software frameworks? We conduct a large-scale study of the portability of mainstream ML frameworks across different hardware types. Our findings paint an uncomfortable picture -- frameworks can lose more than 40% of their key functions when ported to other hardware. Worse, even when functions are portable, the slowdown in their performance can be extreme and render performance untenable. Collectively, our results reveal how costly straying from a narrow set of hardware-software combinations can be - and suggest that specialization of hardware impedes innovation in machine learning research.