SemiKong: Curating, Training, and Evaluating A Semiconductor Industry-Specific Large Language Model

TL;DR

SemiKong is a pioneering semiconductor-specific large language model that leverages curated industry data and expert knowledge integration to outperform general-purpose models in domain-specific tasks, enabling advanced AI applications in semiconductor manufacturing and design.

Contribution

The paper introduces SemiKong, the first industry-specific LLM for semiconductors, with a comprehensive dataset, expert knowledge integration framework, and demonstrated superior performance over general models.

Findings

SemiKong outperforms larger general-purpose LLMs in semiconductor tasks.

Curated domain-specific dataset enhances model understanding of semiconductor physics.

Framework for integrating expert knowledge improves evaluation of domain-specific models.

Abstract

Large Language Models (LLMs) have demonstrated the potential to address some issues within the semiconductor industry. However, they are often general-purpose models that lack the specialized knowledge needed to tackle the unique challenges of this sector, such as the intricate physics and chemistry of semiconductor devices and processes. SemiKong, the first industry-specific LLM for the semiconductor domain, provides a foundation that can be used to develop tailored proprietary models. With SemiKong 1.0, we aim to develop a foundational model capable of understanding etching problems at an expert level. Our key contributions include (a) curating a comprehensive corpus of semiconductor-related texts, (b) creating a foundational model with in-depth semiconductor knowledge, and (c) introducing a framework for integrating expert knowledge, thereby advancing the evaluation process of domain-specific AI models. Through fine-tuning a pre-trained LLM using our curated dataset, we have shown that SemiKong outperforms larger, general-purpose LLMs in various semiconductor manufacturing and design tasks. Our extensive experiments underscore the importance of developing domain-specific LLMs as a foundation for company- or tool-specific proprietary models, paving the way for further research and applications in the semiconductor domain. Code and dataset will be available at https://github.com/aitomatic/semikong