Accelerating Detailed Routing Convergence through Offline Reinforcement Learning

TL;DR

This paper introduces a reinforcement learning approach to accelerate detailed routing in physical design by dynamically adjusting cost weights, achieving faster convergence and better results than traditional static methods.

Contribution

It presents a novel reinforcement learning method that dynamically adapts routing cost weights, significantly reducing runtime and improving design rule violations in detailed routing.

Findings

Achieves 1.56x faster runtime on ISPD19 benchmarks

Up to 3.01x speedup while maintaining or improving DRV count

Shows signs of generalization across different technologies

Abstract

Detailed routing remains one of the most complex and time-consuming steps in modern physical design due to the challenges posed by shrinking feature sizes and stricter design rules. Prior detailed routers achieve state-of-the-art results by leveraging iterative pathfinding algorithms to route each net. However, runtimes are a major issue in detailed routers, as converging to a solution with zero design rule violations (DRVs) can be prohibitively expensive. In this paper, we propose leveraging reinforcement learning (RL) to enable rapid convergence in detailed routing by learning from previous designs. We make the key observation that prior detailed routers statically schedule the cost weights used in their routing algorithms, meaning they do not change in response to the design or technology. By training a conservative Q-learning (CQL) model to dynamically select the routing cost weights which minimize the number of algorithm iterations, we find that our work completes the ISPD19 benchmarks with 1.56x average and up to 3.01x faster runtime than the baseline router while maintaining or improving the DRV count in all cases. We also find that this learning shows signs of generalization across technologies, meaning that learning designs in one technology can translate to improved outcomes in other technologies.