Mind the Gap: Revealing Inconsistencies Across Heterogeneous AI Accelerators

TL;DR

This study empirically investigates the inconsistencies in machine learning model execution across various AI accelerators from different vendors, revealing significant disparities in support, output, and reliability.

Contribution

It provides the first comprehensive empirical analysis of divergence issues across heterogeneous AI accelerators, highlighting implementation flaws and platform-specific challenges.

Findings

Newer platforms support fewer operators and have higher output discrepancy rates.

Platforms exhibit differences in operator implementation and numerical handling.

Identified multiple implementation flaws and platform-specific issues.

Abstract

While NVIDIA remains the dominant provider of AI accelerators within cloud data center, emerging vendors such as AMD, Intel, Mac, and Huawei offer cost-effective alternatives with claims of compatibility and performance. This paper presents the first empirical study investigating divergence in machine learning model across heterogeneous AI accelerators. Utilizing an automated pipeline, we synthesize over 100,000 variant models derived from 4,000 real-world models and execute them across five different enterprise-grade accelerators. Our findings suggest that newer AI platforms from Mac and Huawei support at least 17\% fewer operators than NVIDIA. These platforms also exhibit a higher rate of output discrepancies (exceeding 5\%), which stem from differences in operator implementations, handling of exceptional numerical values, and instruction scheduling. They are also more susceptible to failures during model compilation-based acceleration, and in some cases, the compiled models produce outputs that differ noticeably from those generated using the standard execution mode. In addition, we identify 7 implementation flaws in PyTorch and 40 platform-specific issues across vendors. These results underscore the challenges of achieving consistent machine learning behavior in an increasingly diverse hardware ecosystem.