M3: Mamba-assisted Multi-Circuit Optimization via MBRL with Effective Scheduling

TL;DR

M3 introduces a novel model-based reinforcement learning approach using Mamba architecture and effective scheduling to optimize multiple analog circuits efficiently, reducing computational costs and enhancing adaptability.

Contribution

It is the first to combine Mamba architecture with MBRL and scheduling for multi-circuit optimization, improving sample efficiency and generalization.

Findings

Significantly improved sample efficiency over existing RL methods.

Effective scheduling enhances training efficiency and convergence.

Enables multi-circuit optimization with distinct parameters and specifications.

Abstract

Recent advancements in reinforcement learning (RL) for analog circuit optimization have demonstrated significant potential for improving sample efficiency and generalization across diverse circuit topologies and target specifications. However, there are challenges such as high computational overhead, the need for bespoke models for each circuit. To address them, we propose M3, a novel Model-based RL (MBRL) method employing the Mamba architecture and effective scheduling. The Mamba architecture, known as a strong alternative to the transformer architecture, enables multi-circuit optimization with distinct parameters and target specifications. The effective scheduling strategy enhances sample efficiency by adjusting crucial MBRL training parameters. To the best of our knowledge, M3 is the first method for multi-circuit optimization by leveraging both the Mamba architecture and a MBRL with effective scheduling. As a result, it significantly improves sample efficiency compared to existing RL methods.