Generative Pre-Training of Time-Series Data for Unsupervised Fault Detection in Semiconductor Manufacturing

TL;DR

This paper presents TRACE-GPT, a novel pre-training approach using transformers and convolutional embeddings for unsupervised fault detection in semiconductor manufacturing time-series data, outperforming previous models.

Contribution

It introduces TRACE-GPT, a new model combining convolutional embeddings and GPT for effective unsupervised fault detection in unlabeled semiconductor sensor data.

Findings

Achieves highest F1 score at EER across datasets

Outperforms previous unsupervised models

Close to supervised state-of-the-art performance

Abstract

This paper introduces TRACE-GPT, which stands for Time-seRies Anomaly-detection with Convolutional Embedding and Generative Pre-trained Transformers. TRACE-GPT is designed to pre-train univariate time-series sensor data and detect faults on unlabeled datasets in semiconductor manufacturing. In semiconductor industry, classifying abnormal time-series sensor data from normal data is important because it is directly related to wafer defect. However, small, unlabeled, and even mixed training data without enough anomalies make classification tasks difficult. In this research, we capture features of time-series data with temporal convolutional embedding and Generative Pre-trained Transformer (GPT) to classify abnormal sequences from normal sequences using cross entropy loss. We prove that our model shows better performance than previous unsupervised models with both an open dataset, the University of California Riverside (UCR) time-series classification archive, and the process log of our Chemical Vapor Deposition (CVD) equipment. Our model has the highest F1 score at Equal Error Rate (EER) across all datasets and is only 0.026 below the supervised state-of-the-art baseline on the open dataset.