AstRL: Analog and Mixed-Signal Circuit Synthesis with Deep Reinforcement Learning

TL;DR

This paper introduces AstRL, a deep reinforcement learning framework for analog and mixed-signal circuit synthesis, enabling automated, fine-grained, and structurally valid circuit generation with high success rates.

Contribution

It presents a novel graph-based, RL-driven approach for AMS circuit synthesis that operates at the transistor level, achieving expert-level, generalized design generation validated in simulation.

Findings

100% of generated designs are structurally correct

Over 90% of designs meet specified functionality

Significant improvements over existing methods in design metrics

Abstract

Analog and mixed-signal (AMS) integrated circuits (ICs) lie at the core of modern computing and communications systems. However, despite the continued rise in design complexity, advances in AMS automation remain limited. This reflects the central challenge in developing a generalized optimization method applicable across diverse circuit design spaces, many of which are distinct, constrained, and non-differentiable. To address this, our work casts circuit design as a graph generation problem and introduces a novel method of AMS synthesis driven by deep reinforcement learning (AstRL). Based on a policy-gradient approach, AstRL generates circuits directly optimized for user-specified targets within a simulator-embedded environment that provides ground-truth feedback during training. Through behavioral-cloning and discriminator-based similarity rewards, our method demonstrates, for the first time, an expert-aligned paradigm for generalized circuit generation validated in simulation. Importantly, the proposed approach operates at the level of individual transistors, enabling highly expressive, fine-grained topology generation. Strong inductive biases encoded in the action space and environment further drive structurally consistent and valid generation. Experimental results for three realistic design tasks illustrate substantial improvements in conventional design metrics over state-of-the-art baselines, with 100% of generated designs being structurally correct and over 90% demonstrating required functionality.