Chiplet-Gym: Optimizing Chiplet-based AI Accelerator Design with Reinforcement Learning

TL;DR

This paper introduces Chiplet-Gym, an RL-based framework that optimizes chiplet-based AI accelerator designs by exploring the design space to improve throughput, energy efficiency, and cost-effectiveness.

Contribution

It presents a novel reinforcement learning approach for system and package-level co-design of chiplet-based AI accelerators, addressing the vast design space and cost constraints.

Findings

Achieves 1.52X throughput compared to baseline.

Reduces energy consumption to 0.27X.

Maintains low die cost with minimal package cost increase.

Abstract

Modern Artificial Intelligence (AI) workloads demand computing systems with large silicon area to sustain throughput and competitive performance. However, prohibitive manufacturing costs and yield limitations at advanced tech nodes and die-size reaching the reticle limit restrain us from achieving this. With the recent innovations in advanced packaging technologies, chiplet-based architectures have gained significant attention in the AI hardware domain. However, the vast design space of chiplet-based AI accelerator design and the absence of system and package-level co-design methodology make it difficult for the designer to find the optimum design point regarding Power, Performance, Area, and manufacturing Cost (PPAC). This paper presents Chiplet-Gym, a Reinforcement Learning (RL)-based optimization framework to explore the vast design space of chiplet-based AI accelerators, encompassing the resource allocation, placement, and packaging architecture. We analytically model the PPAC of the chiplet-based AI accelerator and integrate it into an OpenAI gym environment to evaluate the design points. We also explore non-RL-based optimization approaches and combine these two approaches to ensure the robustness of the optimizer. The optimizer-suggested design point achieves 1.52X throughput, 0.27X energy, and 0.01X die cost while incurring only 1.62X package cost of its monolithic counterpart at iso-area.