Validating Monte Carlo simulations for an analysis chain in H.E.S.S

TL;DR

This paper validates and improves Monte Carlo simulations for the H.E.S.S. gamma-ray observatory, ensuring accurate modeling of atmospheric and detector effects to enhance the precision of gamma-ray measurements.

Contribution

It introduces a flexible framework for detailed validation of MC simulations, improving the accuracy of IRFs for H.E.S.S. data analysis.

Findings

Cosmic ray trigger rates matched within a few percent

Instrument response functions achieved percent-level accuracy

Simulation improvements enabled precise gamma-ray source analysis

Abstract

Imaging Air Cherenkov Telescopes (IACTs) detect very high energetic (VHE) gamma rays. They observe the Cherenkov light emitted in electromagnetic shower cascades that gamma rays induce in the atmosphere. A precise reconstruction of the primary photon energy and the source flux depends heavily on accurate Monte Carlo (MC) simulations of the shower propagation and the detector response, and therefore also on adequate assumptions about the atmosphere at the site and time of a measurement. Here, we present the results of an extensive validation of the MC simulations for an analysis chain of the H.E.S.S. experiment with special focus on the recently installed FlashCam camera on the large 28 m telescope. One goal of this work was to create a flexible and easy-to-use framework to facilitate the detailed validation of MC simulations also for past and future phases of the H.E.S.S. experiment. Guided by the underlying physics, the detector simulation and the atmospheric transmission profiles were gradually improved until low level parameters such as cosmic ray (CR) trigger rates matched within a few percent between simulations and observational data. This led to instrument response functions (IRFs) with which the analysis of current H.E.S.S. data can ultimately be carried out within percent accuracy, substantially improving earlier simulations.

Tables

Table 1. Table 1 : Change of simulation parameters and their effects on the CT1-4 stereo participation and CT5 mono trigger rates. If the changes and effects differ between CT1-4 the average value is quoted. Presented are the absolute variations and their absolute effects on the trigger rates.

Parameter	Change CT1-4	Effect CT1-4	Change CT5	Effect CT5
Aerosol level	$50\%$	$10\%$	$50\%$	$12\%$
Trigger threshold	$27\%$	$41\%$	$4\%$	$6\%$
Mirror reflectivity	$2.4\%$	$15\%$	$8\%$	$23\%$
NSB	$67\%$	$3-6\%$	$55\%$	$6\%$
PSF80	$20\%$	$1\%$	$9\%$

Equations

$${\text{PSF}}_{80}(\Theta )=\sqrt{{\text{R}}_{\text{min}}^2+{\text{d}}_1^2\cdot (\sin (\Theta )-\sin ({\Theta }_0){)}^2+{\text{d}}_2^2\cdot (\cos (\Theta )-\cos ({\Theta }_0){)}^2}$$

Full text

Here, we present the results of an extensive validation of the MC simulations for an analysis chain of the H.E.S.S. experiment with special focus on the recently installed FlashCam camera on the large $28\text{\,}\m$ telescope. One goal of this work was to create a flexible and easy-to-use framework to facilitate the detailed validation of MC simulations also for past and future phases of the H.E.S.S. experiment.

Guided by the underlying physics, the detector simulation and the atmospheric transmission profiles were gradually improved until low level parameters such as cosmic ray (CR) trigger rates matched within a few percent between simulations and observational data. This led to instrument response functions (IRFs) with which the analysis of current H.E.S.S. data can ultimately be carried out within percent accuracy, substantially improving earlier simulations.

1 Introduction

MC simulations are crucial for the analysis of data taken by IACTs including the creation of IRFs for these instruments. A prominent toolkit for IACT array simulations is sim_telarray [1] in connection with CORSIKA [2]. This combination is used for MC simulations within one analysis chain of the H.E.S.S. experiment and also for studies of the upcoming Cherenkov Telescope Array (CTA). In this work, we present the validation of simulations for an analysis chain of the H.E.S.S. array. It consists of CT5, a large central telescope ($28\text{\,}\mathrm{m}$ mirror diameter), surrounded by four small telescopes CT1-4 ($12\text{\,}\mathrm{m}$ mirror diameter). Focus is put on the FlashCam camera that was installed at the end of 2019 on CT5 [3], [4]. The goal is to reach consistency between MC simulations and observational data up to the DL3111For more information see https://doi.org/10.3390/universe7100374. data level. Repeatability is another important goal of this work which is achieved through a Python code base. Within these proceedings, we report on the various steps of the validation, learned lessons and reached improvements.

2 Basic MC and single telescope validation

2.1 Basic MC checks

In a first step, single telescope simulations, using a laser with a fixed photon intensity as simulated light source, were used to check the consistency of the charge integration algorithms and the photo electron definition. Thereby, consistency within a few percent between simulations and measurements could be established.

To validate the optical Point Spread Function (PSF) adopted in simulations for each telescope, we use the included ray-tracing feature of sim_telarray which calculates the path of photons through the optical system. The response of the system is evaluated using parallel light as an input. The telescope is pointed at an infinitely distant star at zenith. The off-axis response is probed by introducing a pointing shift between the telescope and the simulated star. An intensity map is created from the photon positions on the camera-lid and a circle is inscribed around the centre of gravity. The radius of this circle is increased until 80 % of the total signal is contained within. This 80 % containment radius is further referred to as the PSF80 of the telescope.

The PSF80 is simulated for different telescope elevations $\Theta$ and compared to measurements obtained within the same hardware phase222That is a period of time in which the hardware configuration and instrument response is considered constant. A new phase starts e.g. when hardware is replaced, mirrors are cleaned, software settings are changed, etc.. The simulated PSF80 in sim_telarray is evaluated using the following equation:

[TABLE]

Here, $\text{R}_{\text{min}}$, $\text{d}_{1}$, $\text{d}_{2}$ and $\Theta_{0}$ are free parameters that can be specified in the simulation configuration. These are readjusted until the overall deviation between simulated and measured PSF80 is minimized. With the new telescope dependent parameters the PSF80 deviation is reduced to $<5\,\%\approx 0.4\,\text{mm}$ in the focal plane (compare Figure 1) which is deemed acceptable.

2.2 Single telescope consistency

To understand whether the properties of the single telescopes in an array are defined correctly in simulations, parameters such as the trigger rates, the raw pixel intensity, and the contribution of the pedestal as well as its noise need to be validated for each telescope individually.

The measured trigger rates are dominated by hadronic showers. Hence, the investigation of the trigger rates is conducted via proton simulations, applying a correction factor for heavier nuclei, which has been validated beforehand with dedicated simulations to be $\approx 1.34$ for the trigger rates of the entire H.E.S.S. array. The raw trigger rates are calculated by folding the effective area with a CR proton spectrum. For high accuracy this work uses the Global Spline Fit spectrum by Hans Dembinsky et al. [6] as it also considers spectral features deviating from a simple power law. The effective area is calculated by taking into account the amount of simulated and triggered showers, simulated area and solid angle. One has to keep in mind that H.E.S.S. is operated in a hybrid fashion with CT1-5 triggering in stereo and additionally CT5 in mono mode. Hence, for CT1-4 the stereo participation rate and for CT5 the mono rate are the prime quantities to compare [7], [3].

In addition to the instrument-specific settings (i.e. trigger threshold, Night Sky Background (NSB) values, PSF80 and reflectivity) some general simulation settings (i.e. simulated energy range, view cone, photon bunch size333Simulated Cherenkov photons are stored in so called photon bunches. A bunch size of five is chosen as a compromise between computing time and accuracy. and atmospheric transmission profile) were adjusted to match real measurements. The various adjustments and their effects on the telescope trigger rates are summarised in Table 1.

The resulting trigger rates can be seen in Figure 2 as a function of zenith angle. For one representative of each of the two H.E.S.S. telescope types, we show both the simulated and real trigger rates. The uncertainty intervals show the investigated systematic uncertainties. For the mirror reflectivity (derived from muon simulations [8]) we accepted an uncertainty of $2\,\%$ leading to a systematic uncertainty of $\approx 12\,\%$ in the trigger rates. The systematic uncertainty arising from the choice of the interaction model in the CORSIKA simulation was investigated by repeating the same simulation set with the models QGSJET-2, EPOS, and SYBILL [2], [9]. A variation of $\approx 4\,\%$ in the resulting trigger rates was found. In addition, the assumed systematic uncertainty on these rates for the implementation of H.E.S.S. in sim_telarray is $10\,\%$.

The detector (in H.E.S.S.: photo-multiplier tubes) output in the cameras is offset by the pedestal. While measurements are corrected for the mean of this value, its standard deviation (’pedestal width’) contributes as noise, mostly caused by diffuse NSB light [10]. Under given observing conditions444E.g., phase and position of the moon, weather and atmospheric conditions, pointing direction, etc., the amount of NSB-photons reaching the cameras depends primarily on the optical brightness of the observed field. Here, differences of a factor of 4 or greater are possible [11]. Therefore, we adjust the pedestal width of the simulations to the median555Mean is discouraged here, as it would be biased by high-value outliers, e.g. from pixels with a star in their Field of View (FoV) that then are automatically switched off. of observations’ pedestal width. For this adjustment, the NSB should be set to a reasonable value consistent with calculations that reproduces the required pedestal width in simulations. This calculation entails folding the NSB flux at the site of the observatory with the collection area of the telescope and the pixel FoV. Provided an NSB value, sim_telarray handles the mirror and quantum efficiencies. However, possible reflections of NSB light from the ground back into the camera must be taken into account manually. Typical NSB levels are of the order $0.1\text{\,}\mathrm{GHz}$ for CT1-4 and $0.3\text{\,}\mathrm{GHz}$ for CT5.

3 Higher level and array validation

3.1 Cleaning and background

In the analysis chain validated, shower images are cleaned after all calibration steps to avoid any major influence of noise on the image reconstruction. In H.E.S.S., cleaning is done using the tail cut method [12]. From the cleaned images the Hillas parameters [13] are derived to describe the image properties. To test the proper description of the hadronic background the image properties of proton simulations are compared to those of observation runs with no strong gamma source in the FoV (off-runs). The proton simulations were weighted by energy to represent the measured CR proton spectrum [6]. The off-runs were selected to have similar observing conditions as the simulation set, especially in terms of NSB, atmospheric transparency and zenith angle. As an example Figure 3 shows the Hillas length divided by shower intensity distributions for CT1 and CT5. There is a good agreement between data and simulations, that is also reflected in other image parameters. Thereby we conclude that the data-simulation consistency persists also after cleaning. This is crucial as the cleaned images are the basis for the energy & direction reconstruction in Hillas based analysis algorithms as well as for the gamma-hadron separation.

3.2 High level validation with the Crab Nebula spectrum

Energy and direction reconstruction techniques as well as the gamma-hadron separation rely on matching gamma ray simulations. Further, instrument response functions such as effective area, energy dispersion matrix and point spread function are produced from them. To cross check the overall high level performance it is crucial to eventually compare the derived spectrum to a known steady source. For this purpose, the Crab Nebula is selected as the typical standard candle in VHE astrophysics. Its spectrum was approximated by a power-law model of the form $\upphi_{0}\left(\frac{E}{E_{0}}\right)^{-\Gamma}$ to be comparable with the published H.E.S.S.spectrum [12]. It can be derived under different atmospheric conditions using simulations including correspondingly different atmospheric transmission profiles. This was done already as a cross-check in the published paper on the nova RS Ophiuchi [14], for which the monoscopic analysis was based on the simulation configuration derived in this work. The resulting spectral parameters are shown in Figure 4. The monoscopic flux normalisation is consistent with respect to an atmosphere corrected stereo analysis on the same data sets as well as the reference Crab spectrum from [12] within $15\,\%$. The index shows a systematic shift by $\sim 0.2$ that can be explained by the observed hardening of the Crab spectrum towards lower energies. This behaviour becomes more important for the mono analysis with its lower energy threshold.

4 Conclusion and outlook

We present a procedure to systematically validate all steps of a Monte Carlo simulation configuration for an array of IACTs by comparing simulated to measured properties. It was applied to an analysis chain of the H.E.S.S. experiment with great success. Using the presented method, simulations for future hardware iterations or phases can be adapted and validated with little effort.

The presented procedures will be of use for the upcoming CTA observatory as well. When operational, it is possible to quickly assess whether the assumptions made in current MC simulation configurations are correct. Further, a validation of the MC configurations will be necessary e.g. when the hardware of CTA will be updated in the future. Care was taken that the entire framework is well documented for future application.

Additionally, to the procedure for validating MC configurations, we developed a scheme to assess the impact of short term changes in the atmospheric transmission on the response of atmospheric Cherenkov detectors and to finally correct for them. This scheme is described in detail in a separate work [16]. It allows to use more of the data obtained by Cherenkov detectors than before and to have a more reliable estimation of primary photon energies.