AnalogSeeker: An Open-source Foundation Language Model for Analog Circuit Design

TL;DR

AnalogSeeker is an open-source foundation language model tailored for analog circuit design, integrating domain knowledge through a novel corpus collection and knowledge distillation approach, achieving high accuracy and practical design effectiveness.

Contribution

The paper introduces a new open-source foundation model for analog circuits, with a domain-specific corpus, granular knowledge distillation, and a fine-tuning training paradigm, advancing analog design AI.

Findings

Achieves 85.04% accuracy on AMSBench-TQA benchmark.

Outperforms original models by 15.67 percentage points.

Effective in downstream operational amplifier design tasks.

Abstract

In this paper, we propose AnalogSeeker, an effort toward an open-source foundation language model for analog circuit design, with the aim of integrating domain knowledge and giving design assistance. To overcome the scarcity of data in this field, we employ a corpus collection strategy based on the domain knowledge framework of analog circuits. High-quality, accessible textbooks across relevant subfields are systematically curated and cleaned into a textual domain corpus. To address the complexity of knowledge of analog circuits, we introduce a granular domain knowledge distillation method. Raw, unlabeled domain corpus is decomposed into typical, granular learning nodes, where a multi-agent framework distills implicit knowledge embedded in unstructured text into question-answer data pairs with detailed reasoning processes, yielding a fine-grained, learnable dataset for fine-tuning. To address the unexplored challenges in training analog circuit foundation models, we explore and share our training methods through both theoretical analysis and experimental validation. We finally establish a fine-tuning-centric training paradigm, customizing and implementing a neighborhood self-constrained supervised fine-tuning algorithm. This approach enhances training outcomes by constraining the perturbation magnitude between the model's output distributions before and after training. In practice, we train the Qwen2.5-32B-Instruct model to obtain AnalogSeeker, which achieves 85.04% accuracy on AMSBench-TQA, the analog circuit knowledge evaluation benchmark, with a 15.67% point improvement over the original model and is competitive with mainstream commercial models. Furthermore, AnalogSeeker also shows effectiveness in the downstream operational amplifier design task. AnalogSeeker is open-sourced at https://huggingface.co/analogllm/analogseeker for research use.