UNIFERENCE: A Discrete Event Simulation Framework for Developing Distributed AI Models

TL;DR

UNIFERENCE is a versatile discrete-event simulation framework that models distributed AI systems, enabling seamless transition from simulation to real deployment with high accuracy and reproducibility.

Contribution

It introduces a unified simulation environment for developing and benchmarking distributed AI models, integrating with PyTorch and supporting diverse hardware configurations.

Findings

Profiles runtime with up to 98.6% accuracy compared to real deployments.

Models device and network behavior without rollbacks, preserving causal order.

Provides an open-source platform bridging simulation and deployment for distributed AI.

Abstract

Developing and evaluating distributed inference algorithms remains difficult due to the lack of standardized tools for modeling heterogeneous devices and networks. Existing studies often rely on ad-hoc testbeds or proprietary infrastructure, making results hard to reproduce and limiting exploration of hypothetical hardware or network configurations. We present UNIFERENCE, a discrete-event simulation (DES) framework designed for developing, benchmarking, and deploying distributed AI models within a unified environment. UNIFERENCE models device and network behavior through lightweight logical processes that synchronize only on communication primitives, eliminating rollbacks while preserving the causal order. It integrates seamlessly with PyTorch Distributed, enabling the same codebase to transition from simulation to real deployment. Our evaluation demonstrates that UNIFERENCE profiles runtime with up to 98.6% accuracy compared to real physical deployments across diverse backends and hardware setups. By bridging simulation and deployment, UNIFERENCE provides an accessible, reproducible platform for studying distributed inference algorithms and exploring future system designs, from high-performance clusters to edge-scale devices. The framework is open-sourced at https://github.com/Dogacel/Uniference.