Soft Sensing Transformer: Hundreds of Sensors are Worth a Single Word

TL;DR

This paper introduces a Soft Sensing Transformer model that effectively processes high-dimensional industrial sensor data, outperforming traditional models like auto-encoders and LSTMs, and provides large-scale datasets for the field.

Contribution

It presents the first benchmarking of transformer models on large-scale industrial soft sensing data, demonstrating superior performance over existing methods.

Findings

Transformer outperforms auto-encoder and LSTM models in soft sensing tasks.

Provides large-scale, high-dimensional sensor datasets for industrial applications.

First to benchmark transformer models on such data in academia and industry.

Abstract

With the rapid development of AI technology in recent years, there have been many studies with deep learning models in soft sensing area. However, the models have become more complex, yet, the data sets remain limited: researchers are fitting million-parameter models with hundreds of data samples, which is insufficient to exercise the effectiveness of their models and thus often fail to perform when implemented in industrial applications. To solve this long-lasting problem, we are providing large scale, high dimensional time series manufacturing sensor data from Seagate Technology to the public. We demonstrate the challenges and effectiveness of modeling industrial big data by a Soft Sensing Transformer model on these data sets. Transformer is used because, it has outperformed state-of-the-art techniques in Natural Language Processing, and since then has also performed well in the direct application to computer vision without introduction of image-specific inductive biases. We observe the similarity of a sentence structure to the sensor readings and process the multi-variable sensor readings in a time series in a similar manner of sentences in natural language. The high-dimensional time-series data is formatted into the same shape of embedded sentences and fed into the transformer model. The results show that transformer model outperforms the benchmark models in soft sensing field based on auto-encoder and long short-term memory (LSTM) models. To the best of our knowledge, we are the first team in academia or industry to benchmark the performance of original transformer model with large-scale numerical soft sensing data.