RACE: A Reinforcement Learning Framework for Improved Adaptive Control of NoC Channel Buffers

TL;DR

RACE is a reinforcement learning framework designed to optimize adaptive buffer control in Network-on-Chip architectures, significantly reducing latency and energy consumption by intelligently managing RMC buffers based on congestion awareness.

Contribution

It introduces a novel RL-based approach with a new reward metric for scalable, congestion-aware buffer control in NoC architectures, outperforming existing policies.

Findings

Reduces NoC latency by up to 48.9%

Cuts energy consumption by up to 47.1%

Outperforms state-of-the-art buffer control policies

Abstract

Network-on-chip (NoC) architectures rely on buffers to store flits to cope with contention for router resources during packet switching. Recently, reversible multi-function channel (RMC) buffers have been proposed to simultaneously reduce power and enable adaptive NoC buffering between adjacent routers. While adaptive buffering can improve NoC performance by maximizing buffer utilization, controlling the RMC buffer allocations requires a congestion-aware, scalable, and proactive policy. In this work, we present RACE, a novel reinforcement learning (RL) framework that utilizes better awareness of network congestion and a new reward metric ("falsefulls") to help guide the RL agent towards better RMC buffer control decisions. We show that RACE reduces NoC latency by up to 48.9%, and energy consumption by up to 47.1% against state-of-the-art NoC buffer control policies.