Efficient Bypass in Mesh and Torus NoCs

TL;DR

This paper introduces NEBB, a novel bypass mechanism for NoC routers that allows bypassing even with non-empty buffers, significantly reducing latency and power consumption in mesh and torus topologies.

Contribution

The paper proposes NEBB, a new bypass technique that operates with non-empty buffers, extending to torus topologies and combining flow control strategies for improved performance.

Findings

Up to 75% reduction in buffered flits for single-flit packets

Latency and power reductions of up to 30% and 23% respectively

Bypass utilization remains high and stable across different VC configurations

Abstract

Minimizing latency and power are key goals in the design of NoC routers. Different proposals combine lookahead routing and router bypass to skip the arbitration and buffering, reducing router delay. However, the conditions to use them requires completely empty buffers in the intermediate routers. This restricts the amount of flits that use the bypass pipeline especially at medium and high loads, increasing latency and power. This paper presents NEBB, Non-Empty Buffer Bypass, a mechanism that allows to bypass flits even if the buffers to bypass are not empty. The mechanism applies to wormhole and virtual-cut-through, each of them with different advantages. NEBB-Hybrid is proposed to employ the best flow control in each situation. The mechanism is extended to torus topologies, using FBFC and shared buffers. The proposals have been evaluated using Booksim, showing up to 75% reduction of the buffered flits for single-flit packets, which translates into latency and dynamic power reductions of up to 30% and 23% respectively. For bimodal traffic, these improvements are 20 and 21% respectively. Additionally, the bypass utilization is largely independent of the number of VCs when using shared buffers and very competitive with few private ones, allowing to simplify the allocation mechanisms.