Trust-Region Method with Deep Reinforcement Learning in Analog Design Space Exploration

TL;DR

This paper presents a novel trust-region approach using model-based deep reinforcement learning for efficient analog design space exploration, achieving significant improvements over traditional methods and surpassing human performance.

Contribution

It introduces a model-based reinforcement learning framework with trust-region strategy for analog design, accommodating PVT variations and enabling industrial deployment.

Findings

Orders of magnitude reduction in search iterations

Surpasses human designer performance on TSMC 5/6nm circuits

Framework is actively used in industrial settings

Abstract

This paper introduces new perspectives on analog design space search. To minimize the time-to-market, this endeavor better cast as constraint satisfaction problem than global optimization defined in prior arts. We incorporate model-based agents, contrasted with model-free learning, to implement a trust-region strategy. As such, simple feed-forward networks can be trained with supervised learning, where the convergence is relatively trivial. Experiment results demonstrate orders of magnitude improvement on search iterations. Additionally, the unprecedented consideration of PVT conditions are accommodated. On circuits with TSMC 5/6nm process, our method achieve performance surpassing human designers. Furthermore, this framework is in production in industrial settings.