Towards Foundation Models for Experimental Readout Systems Combining Discrete and Continuous Data

TL;DR

This paper introduces a novel foundation model for nuclear physics detector data that combines discrete and continuous data processing, enabling high-fidelity sequence generation and effective particle identification.

Contribution

It proposes four innovative techniques for integrating discrete and continuous data in a foundation model, improving resolution and conditional generation capabilities.

Findings

Model achieves high-fidelity Cherenkov photon sequence generation

Enables effective particle identification and noise filtering

Demonstrates successful generalization to reconstruction tasks

Abstract

We present a (proto) Foundation Model for Nuclear Physics, capable of operating on low-level detector inputs from Imaging Cherenkov Detectors at the future Electron Ion Collider. Building upon established next-token prediction approaches, we aim to address potential challenges such as resolution loss from existing tokenization schemes and limited support for conditional generation. We propose four key innovations: (i) separate vocabularies for discrete and continuous variates, combined via Causal Multi-Head Cross-Attention (CMHCA), (ii) continuous kinematic conditioning through prepended context embeddings, (iii) scalable and simple, high-resolution continuous variate tokenization without joint vocabulary inflation, and (iv) class conditional generation through a Mixture of Experts. Our model enables fast, high-fidelity generation of pixel and time sequences for Cherenkov photons, validated through closure tests in the High Performance DIRC. We also show our model generalizes to reconstruction tasks such as pion/kaon identification, and noise filtering, in which we show its ability to leverage fine-tuning under specific objectives.