Accelerating Precise End-to-End Simulation: Latency-Sensitive Many-core System Modeling

TL;DR

This paper introduces a fast, accurate end-to-end simulation framework for latency-sensitive many-core architectures, enabling scalable performance modeling and hardware/software optimization with significantly reduced simulation time.

Contribution

It presents a novel modeling approach that captures latency-sensitive interconnect behavior in large-scale many-core systems, achieving high accuracy and speedup over traditional RTL simulation.

Findings

Model achieves below 7% error compared to RTL gold standard.

Simulation is up to 115 times faster than cycle-accurate RTL.

Framework enables effective hardware/software co-design and optimization.

Abstract

Modern large language model workloads put increasing demands on parallel compute capability and on-chip memory capacity, while also stressing fine-grained data movement and synchronization. These trends motivate exploring and designing many-core accelerators with tightly coupled scratchpad memory (SPM) for scalable compute and predictable, explicitly managed data access. However, this architectural shift raises two challenges: cycle-accurate register-transfer level (RTL) simulation becomes prohibitively slow as system complexity grows, and performance estimation requires precise modeling of latency-sensitive interconnect behavior. This paper presents a fast yet accurate end-to-end modeling approach for latency-sensitive many-core architectures, targeting large-scale instances such as TeraNoC with 1024 cores and a 4MiB globally shared L1 SPM. The approach captures timing behavior of latency-sensitive SPM accesses across multiple interconnect scales, while abstracting non-essential hardware details. Across diverse benchmarks, the model tracks a cycle-accurate RTL golden model with errors below 7%, while delivering up to 115x faster simulation. The framework also provides detailed profiling across processing elements and interconnect, enabling efficient end-to-end software development and hardware design exploration. Two case studies demonstrate its practicality: profiling-guided optimization of FlashAttention-2 to reduce interconnect stalls and synchronization overhead, and design space exploration of network-on-chip (NoC) router remapping to alleviate traffic imbalance and improve throughput.