Testing a Novel Self-Assembling Data Paradigm in the Context of IACT Data

TL;DR

This paper introduces a biologically inspired self-assembly data paradigm for event-building in high-energy physics and astronomy, using metadata-based bonds and a dynamic database to improve fault tolerance and data association.

Contribution

It presents a novel self-assembly paradigm for event-building that models data packets as molecules, enabling dynamic, fault-tolerant data association using a fluid database approach.

Findings

Prototype tests with VERITAS data demonstrate feasibility.

Self-assembly improves fault tolerance in high-throughput data scenarios.

Dynamic bonding process enhances data association accuracy.

Abstract

The process of gathering and associating data from multiple sensors or sub-detectors due to a common physical event (the process of event-building) is used in many fields, including high-energy physics and $\gamma$-ray astronomy. Fault tolerance in event-building is a challenging problem that increases in difficulty with higher data throughput rates and increasing numbers of sub-detectors. We draw on biological self-assembly models in the development of a novel event-building paradigm that treats each packet of data from an individual sensor or sub-detector as if it were a molecule in solution. Just as molecules are capable of forming chemical bonds, "bonds" can be defined between data packets using metadata-based discriminants. A database -- which plays the role of a beaker of solution -- continually selects pairs of assemblies at random to test for bonds, which allows single packets and small assemblies to aggregate into larger assemblies. During this process higher-quality associations supersede spurious ones. The database thereby becomes fluid, dynamic, and self-correcting rather than static. We will describe tests of the self-assembly paradigm using our first fluid database prototype and data from the VERITAS $\gamma$-ray telescope.