Self-Supervised Learning for Modeling Gamma-ray Variability in Blazars

TL;DR

This paper introduces a self-supervised Transformer model to analyze gamma-ray variability in blazars, capturing complex stochastic patterns and accommodating data uncertainties, with initial tests showing no time-reversal asymmetry.

Contribution

The work presents a novel application of self-supervised Transformers for modeling blazar gamma-ray variability, handling measurement errors and missing data effectively.

Findings

Model predicts flux distribution quantiles at each time step.

Preliminary analysis finds no significant time-reversal asymmetry.

Demonstrates potential for uncovering structure in complex variability patterns.

Abstract

Blazars are active galactic nuclei with relativistic jets pointed almost directly at Earth. Blazars are characterized by strong, apparently stochastic flux variability at virtually all observed wavelengths and timescales, from minutes to years, the physical origin of which is still poorly understood. In the high-energy gamma-ray band, the Large Area Telescope aboard the Fermi space telescope (Fermi-LAT) has conducted regular monitoring of thousands of blazars since 2008. Deep learning can help uncover structure in gamma-ray blazars' complex variability patterns that traditional methods based on parametric statistical modeling or manual feature engineering may miss. In this work, we propose using a self-supervised Transformer encoder architecture to construct an effective representation of blazar gamma-ray variability. Measurement errors, upper limits, and missing data are accommodated using learned encodings. The model predicts a set of quantiles for the flux probability distribution at each time step, an architecture naturally suited for describing data generated by a stochastic process. As a proof of concept for how the model output can be analyzed to extract scientifically relevant information, a preliminary search for weekly-timescale time-reversal asymmetry in gamma-ray blazar light curves was conducted, finding no significant evidence for asymmetry.