A Predictive Model Based on Transformer with Statistical Feature Embedding in Manufacturing Sensor Dataset

TL;DR

This paper introduces a Transformer-based predictive model with statistical feature embedding and window positional encoding, improving fault detection and virtual metrology in manufacturing sensor data with limited samples.

Contribution

It presents a novel Transformer model that integrates statistical feature embedding and window positional encoding for manufacturing sensor data analysis.

Findings

Outperforms baseline models in fault detection.

Effective with limited sample sizes.

Applicable across various manufacturing industries.

Abstract

In the manufacturing process, sensor data collected from equipment is crucial for building predictive models to manage processes and improve productivity. However, in the field, it is challenging to gather sufficient data to build robust models. This study proposes a novel predictive model based on the Transformer, utilizing statistical feature embedding and window positional encoding. Statistical features provide an effective representation of sensor data, and the embedding enables the Transformer to learn both time- and sensor-related information. Window positional encoding captures precise time details from the feature embedding. The model's performance is evaluated in two problems: fault detection and virtual metrology, showing superior results compared to baseline models. This improvement is attributed to the efficient use of parameters, which is particularly beneficial for sensor data that often has limited sample sizes. The results support the model's applicability across various manufacturing industries, demonstrating its potential for enhancing process management and yield.