BatteryBERT for Realistic Battery Fault Detection Using Point-Masked Signal Modeling

TL;DR

This paper introduces BatteryBERT, a novel BERT-inspired framework for fault detection in lithium-ion batteries, leveraging self-supervised pretraining on time-series data to improve classification accuracy and robustness.

Contribution

It extends BERT architecture with a time-series-to-token module and point-MSM pretraining tailored for battery data, enabling effective self-supervised learning for fault detection.

Findings

Achieves an AUROC of 0.945 on real-world data.

Significantly outperforms existing fault detection methods.

Enhances representation quality and classification accuracy.

Abstract

Accurate fault detection in lithium-ion batteries is essential for the safe and reliable operation of electric vehicles and energy storage systems. However, existing methods often struggle to capture complex temporal dependencies and cannot fully leverage abundant unlabeled data. Although large language models (LLMs) exhibit strong representation capabilities, their architectures are not directly suited to the numerical time-series data common in industrial settings. To address these challenges, we propose a novel framework that adapts BERT-style pretraining for battery fault detection by extending the standard BERT architecture with a customized time-series-to-token representation module and a point-level Masked Signal Modeling (point-MSM) pretraining task tailored to battery applications. This approach enables self-supervised learning on sequential current, voltage, and other charge-discharge cycle data, yielding distributionally robust, context-aware temporal embeddings. We then concatenate these embeddings with battery metadata and feed them into a downstream classifier for accurate fault classification. Experimental results on a large-scale real-world dataset show that models initialized with our pretrained parameters significantly improve both representation quality and classification accuracy, achieving an AUROC of 0.945 and substantially outperforming existing approaches. These findings validate the effectiveness of BERT-style pretraining for time-series fault detection.