Transformer-Based Pulse Shape Discrimination in HPGe Detectors with Masked Autoencoder Pre-training

TL;DR

This paper demonstrates that transformer models, especially with masked autoencoder pre-training, significantly improve pulse shape discrimination in HPGe detectors by operating directly on raw waveforms, outperforming traditional methods and enhancing data efficiency.

Contribution

It introduces transformer-based models with self-supervised pre-training for direct waveform analysis in HPGe detectors, showing improved accuracy and data efficiency over existing approaches.

Findings

Transformers outperform GBDT in PSD tasks.

MAE pre-training reduces labeled data needs by 2-4 times.

Transformers show small energy regression bias.

Abstract

Pulse-shape discrimination (PSD) in high-purity germanium (HPGe) detectors is central to rare-event searches such as neutrinoless double-beta decay (0vBB), yet conventional approaches compress each waveform into a small set of summary parameters, potentially discarding information in the full time series that is relevant for classification. We benchmark transformer-based models that operate directly on digitised waveforms using the Majorana Demonstrator AI/ML data release. Models are trained to reproduce the collaboration-provided accept/reject labels for four standard PSD cuts and to regress calibrated energy. We compare supervised training from scratch, masked autoencoder (MAE) self-supervised pre-training followed by fine-tuning, and a feature-based gradient-boosted decision tree (GBDT) baseline. Transformers outperform GBDT across all PSD targets, with the largest gains on the most challenging labels and on the combined PSD-pass definition. MAE pre-training improves sample efficiency, reducing labelled-data requirements by factors of 2-4 in low-label regimes. For energy regression, both transformer variants show a small common underestimation on the test split, while fine-tuning modestly narrows the residual distribution. These results motivate follow-up studies of robustness across detectors and operating conditions and of performance near QBB.