An Open-Source HW-SW Co-Development Framework Enabling Efficient Multi-Accelerator Systems

TL;DR

SNAX is an open-source HW-SW framework that enables efficient, flexible, and high-performance multi-accelerator systems for AI workloads, combining hardware modules and a compiler for rapid deployment.

Contribution

The paper introduces SNAX, a novel hybrid-coupling HW-SW framework with reusable modules and an MLIR-based compiler for efficient multi-accelerator platform development.

Findings

Achieves over 10x neural network performance improvement

Maintains accelerator utilization above 90%

Demonstrates efficiency on a low-power heterogeneous SoC

Abstract

Heterogeneous accelerator-centric compute clusters are emerging as efficient solutions for diverse AI workloads. However, current integration strategies often compromise data movement efficiency and encounter compatibility issues in hardware and software. This prevents a unified approach that balances performance and ease of use. To this end, we present SNAX, an open-source integrated HW-SW framework enabling efficient multi-accelerator platforms through a novel hybrid-coupling scheme, consisting of loosely coupled asynchronous control and tightly coupled data access. SNAX brings reusable hardware modules designed to enhance compute accelerator utilization, and its customizable MLIR-based compiler to automate key system management tasks, jointly enabling rapid development and deployment of customized multi-accelerator compute clusters. Through extensive experimentation, we demonstrate SNAX's efficiency and flexibility in a low-power heterogeneous SoC. Accelerators can easily be integrated and programmed to achieve > 10x improvement in neural network performance compared to other accelerator systems while maintaining accelerator utilization of > 90% in full system operation.