PrefixRL: Optimization of Parallel Prefix Circuits using Deep Reinforcement Learning

TL;DR

This paper introduces PrefixRL, a deep reinforcement learning approach that designs high-performance parallel prefix circuits, outperforming traditional methods in area and delay metrics for large bit-width adders.

Contribution

It presents a novel RL-based method for designing prefix circuits from scratch, integrating synthesis in the learning process, and demonstrating superior performance over existing baselines.

Findings

RL-designed prefix adders have up to 16-30% lower area for same delay.

Agents trained with open-source tools outperform commercial adders.

The approach generalizes to large bit-width circuit design.

Abstract

In this work, we present a reinforcement learning (RL) based approach to designing parallel prefix circuits such as adders or priority encoders that are fundamental to high-performance digital design. Unlike prior methods, our approach designs solutions tabula rasa purely through learning with synthesis in the loop. We design a grid-based state-action representation and an RL environment for constructing legal prefix circuits. Deep Convolutional RL agents trained on this environment produce prefix adder circuits that Pareto-dominate existing baselines with up to 16.0% and 30.2% lower area for the same delay in the 32b and 64b settings respectively. We observe that agents trained with open-source synthesis tools and cell library can design adder circuits that achieve lower area and delay than commercial tool adders in an industrial cell library.