Harnessing Event Sensory Data for Error Pattern Prediction in Vehicles: A Language Model Approach

TL;DR

This paper introduces CarFormer and EPredictor, Transformer-based models that predict vehicle error patterns and their timing using event data, enabling proactive maintenance and improved vehicle safety.

Contribution

It presents a novel self-supervised Transformer approach for predicting vehicle errors and their timing from multivariate event streams, addressing data challenges.

Findings

Achieved 80% F1 score in predicting error types with half the sequence

Forecasted error occurrence time with an average absolute error of 58.4 hours

Demonstrated high predictive accuracy despite data limitations

Abstract

In this paper, we draw an analogy between processing natural languages and processing multivariate event streams from vehicles in order to predict $\textit{when}$ and $\textit{what}$ error pattern is most likely to occur in the future for a given car. Our approach leverages the temporal dynamics and contextual relationships of our event data from a fleet of cars. Event data is composed of discrete values of error codes as well as continuous values such as time and mileage. Modelled by two causal Transformers, we can anticipate vehicle failures and malfunctions before they happen. Thus, we introduce $\textit{CarFormer}$, a Transformer model trained via a new self-supervised learning strategy, and $\textit{EPredictor}$, an autoregressive Transformer decoder model capable of predicting $\textit{when}$ and $\textit{what}$ error pattern will most likely occur after some error code apparition. Despite the challenges of high cardinality of event types, their unbalanced frequency of appearance and limited labelled data, our experimental results demonstrate the excellent predictive ability of our novel model. Specifically, with sequences of $160$ error codes on average, our model is able with only half of the error codes to achieve $80\%$ F1 score for predicting $\textit{what}$ error pattern will occur and achieves an average absolute error of $58.4 \pm 13.2$h $\textit{when}$ forecasting the time of occurrence, thus enabling confident predictive maintenance and enhancing vehicle safety.