Characterization and Photometric Performance of the Hyper Suprime-Cam Software Pipeline

TL;DR

This paper evaluates the photometric accuracy of the Hyper Suprime-Cam pipeline using synthetic galaxies injected into real images, revealing strengths and limitations in star and galaxy photometry at various magnitudes.

Contribution

It introduces SynPipe, a Python framework for synthetic galaxy injection, and assesses the HSC pipeline's photometric performance and biases at different depths.

Findings

Achieves 1% photometric precision for stars at i~19.0 mag

Attains 13% photometric precision for synthetic galaxies at i~20.0 mag

Identifies blending and classification issues affecting photometry and morphology measurements

Abstract

The Subaru Strategic Program (SSP) is an ambitious multi-band survey using the Hyper Suprime-Cam (HSC) on the Subaru telescope. The Wide layer of the SSP is both wide and deep, reaching a detection limit of i~26.0 mag. At these depths, it is challenging to achieve accurate, unbiased, and consistent photometry across all five bands. The HSC data are reduced using a pipeline that builds on the prototype pipeline for the Large Synoptic Survey Telescope. We have developed a Python-based, flexible framework to inject synthetic galaxies into real HSC images called SynPipe. Here we explain the design and implementation of SynPipe and generate a sample of synthetic galaxies to examine the photometric performance of the HSC pipeline. For stars, we achieve 1% photometric precision at i~19.0 mag and 6% precision at i~25.0 in the i-band. For synthetic galaxies with single-Sersic profiles, forced CModel photometry achieves 13% photometric precision at i~20.0 mag and 18% precision at i~25.0 in the i-band. We show that both forced PSF and CModel photometry yield unbiased color estimates that are robust to seeing conditions. We identify several caveats that apply to the version of HSC pipeline used for the first public HSC data release (DR1) that need to be taking into consideration. First, the degree to which an object is blended with other objects impacts the overall photometric performance. This is especially true for point sources. Highly blended objects tend to have larger photometric uncertainties, systematically underestimated fluxes and slightly biased colors. Second, >20% of stars at 22.5< i < 25.0 mag can be misclassified as extended objects. Third, the current CModel algorithm tends to strongly underestimate the half-light radius and ellipticity of galaxy with i>21.5 mag.

Tables

Table 1. Table 1: Summary of precisions and accuracies of forced PSF magnitudes in all five bands and in both goodSeeing (FWHM = 0.45 absent 0.45 =0.45 ′′ ) and badSeeing (FWHM = 0.70 absent 0.70 =0.70 ′′ ) tracts based on the statistics of the difference between output forced PSF magnitude and input value ( Δ ​ mag Δ mag \Delta\mathrm{mag} ) shown in Fig 5 . Here, the precision is described using the statistical uncertainties of Δ ​ mag Δ mag \Delta\mathrm{mag} within a series of input magnitude bins ( σ Δ ​ Mag subscript 𝜎 Δ Mag \sigma_{\Delta\mathrm{Mag}} ). The accuracy of PSF photometry in the same magnitude bin is described by the mean value of Δ ​ mag Δ mag \Delta\mathrm{mag} .

Table: Summary of forced PSF Magnitude
Input	Filter	$g$		$r$		$i$		$z$		$y$
Magnitude	FWHM	$0.45$′′	$0.70$′′	$0.45$′′	$0.70$′′	$0.45$′′	$0.70$′′	$0.45$′′	$0.70$′′	$0.45$′′	$0.70$′′
(mag)		(mag)	(mag)	(mag)	(mag)	(mag)	(mag)	(mag)	(mag)	(mag)	(mag)
19.0	${\sigma }_{\Delta \mathrm{Mag}}$	0.017	0.017	0.021	0.022	0.014	0.011	0.011	0.019	0.016	0.019
		0.008	0.006	0.011	0.009	0.002	0.001	0.002	0.003	0.006	0.007
20.0	${\sigma }_{\Delta \mathrm{Mag}}$	0.018	0.018	0.023	0.025	0.015	0.012	0.012	0.020	0.017	0.019
		0.009	0.005	0.011	0.009	0.002	0.002	0.003	0.003	0.006	0.008
21.0	${\sigma }_{\Delta \mathrm{Mag}}$	0.019	0.018	0.024	0.026	0.017	0.014	0.014	0.021	0.021	0.022
		0.009	0.005	0.012	0.009	0.002	0.002	0.002	0.003	0.004	0.008
22.0	${\sigma }_{\Delta \mathrm{Mag}}$	0.023	0.020	0.025	0.029	0.018	0.015	0.017	0.024	0.035	0.037
		0.009	0.005	0.011	0.009	0.002	0.002	0.003	0.004	0.006	0.006
23.0	${\sigma }_{\Delta \mathrm{Mag}}$	0.031	0.026	0.029	0.031	0.021	0.022	0.025	0.039	0.076	0.079
		0.008	0.004	0.011	0.009	0.002	0.002	0.003	0.005	- 0.001	0.003
24.0	${\sigma }_{\Delta \mathrm{Mag}}$	0.049	0.038	0.036	0.046	0.030	0.046	0.056	0.082	0.147	0.146
		0.007	0.003	0.010	0.007	0.002	0.001	0.005	0.003	- 0.003	- 0.033
25.0	${\sigma }_{\Delta \mathrm{Mag}}$	0.119	0.108	0.087	0.107	0.062	0.103	0.124	0.152	–	–
		- 0.010	- 0.008	0.004	0.006	0.003	- 0.009	- 0.001	- 0.010	–	–

Table 2. Table 2: Summary of precisions and accuracies of forced PSF colors using ( g − i ) 𝑔 𝑖 (g-i) and ( i − y ) 𝑖 𝑦 (i-y) colors and in both goodSeeing (FWHM = 0.45 absent 0.45 =0.45 ′′ ) and badSeeing (FWHM = 0.70 absent 0.70 =0.70 ′′ ) tracts based on the statistics of the difference between output forced PSF color and input value ( Δ ​ Color Δ Color \Delta\mathrm{Color} ) shown in Fig 7 . Here, we describe the precision and accuracy of PSF color measurements using the statistical uncertainties and mean values of Δ ​ Color Δ Color \Delta\mathrm{Color} in bins of input i 𝑖 i band magnitudes.

Table: Summary of forced PSF Colors
$i_{\mathrm{Input}}$	Color	$\left(g-i\right)$		$\left(i-y\right)$
	FWHM	$0.45$′′	$0.70$′′	$0.45$′′	$0.70$′′
(mag)		(mag)	(mag)	(mag)	(mag)
19.0	${\sigma }_{\Delta \mathrm{Color}}$	0.023	0.022	0.018	0.019
		0.007	0.003	- 0.004	- 0.006
20.0	${\sigma }_{\Delta \mathrm{Color}}$	0.028	0.024	0.019	0.019
		0.007	0.003	- 0.004	- 0.006
21.0	${\sigma }_{\Delta \mathrm{Color}}$	0.034	0.031	0.022	0.022
		0.006	0.002	- 0.004	- 0.006
22.0	${\sigma }_{\Delta \mathrm{Color}}$	0.053	0.051	0.031	0.030
		0.004	0.001	- 0.004	- 0.005
23.0	${\sigma }_{\Delta \mathrm{Color}}$	0.079	0.086	0.057	0.058
		- 0.004	- 0.015	- 0.001	- 0.004
24.0	${\sigma }_{\Delta \mathrm{Color}}$	0.116	0.122	0.111	0.112
		- 0.015	- 0.001	0.009	0.004
25.0	${\sigma }_{\Delta \mathrm{Color}}$	0.162	0.185	0.192	0.216
		- 0.044	- 0.018	0.097	0.097

Table 3. Table 3: Summary of precisions and accuracies of forced cModel magnitudes in all five bands and in both goodSeeing (FWHM = 0.45 absent 0.45 =0.45 ′′ ) and badSeeing (FWHM = 0.70 absent 0.70 =0.70 ′′ ) tracts based on the statistics of the difference between output forced cModel magnitude and input value ( Δ ​ mag Δ mag \Delta\mathrm{mag} ) shown in Fig 10 . Other details are the same with Table 1 .

Table: Summary of forced cModel Magnitude
Input	Filter	$g$		$r$		$i$		$z$		$y$
Magnitude	FWHM	$0.45$′′	$0.70$′′	$0.45$′′	$0.70$′′	$0.45$′′	$0.70$′′	$0.45$′′	$0.70$′′	$0.45$′′	$0.70$′′
(mag)		(mag)	(mag)	(mag)	(mag)	(mag)	(mag)	(mag)	(mag)	(mag)	(mag)
20.0	${\sigma }_{\Delta \mathrm{Mag}}$	0.111	0.137	0.197	0.167	0.173	0.154	0.171	0.168	0.179	0.169
		- 0.006	0.018	- 0.036	- 0.024	- 0.023	0.017	- 0.006	0.027	0.011	0.041
21.0	${\sigma }_{\Delta \mathrm{Mag}}$	0.156	0.161	0.151	0.119	0.157	0.157	0.176	0.155	0.169	0.164
		- 0.036	- 0.008	- 0.015	0.023	0.008	0.043	0.043	0.046	0.056	0.052
22.0	${\sigma }_{\Delta \mathrm{Mag}}$	0.161	0.138	0.147	0.142	0.154	0.153	0.165	0.167	0.178	0.169
		- 0.004	0.029	0.033	0.050	0.004	0.040	0.038	0.045	0.044	0.048
23.0	${\sigma }_{\Delta \mathrm{Mag}}$	0.152	0.147	0.167	0.168	0.168	0.174	0.182	0.185	0.197	0.194
		0.021	0.040	0.033	0.036	0.017	0.021	0.018	0.028	0.014	0.020
24.0	${\sigma }_{\Delta \mathrm{Mag}}$	0.176	0.181	0.179	0.189	0.183	0.193	0.201	0.222	0.241	0.253
		0.009	0.028	0.028	0.025	- 0.001	0.003	- 0.002	0.009	- 0.067	- 0.057
25.0	${\sigma }_{\Delta \mathrm{Mag}}$	0.196	0.202	0.211	0.232	0.218	0.246	0.241	0.278	0.308	0.315
		- 0.005	0.012	0.012	0.022	- 0.006	- 0.013	- 0.012	- 0.034	- 0.224	- 0.299

Table 4. Table 4: Summary of precisions and accuracies of forced cModel colors using ( g − i ) 𝑔 𝑖 (g-i) and ( i − y ) 𝑖 𝑦 (i-y) colors and in both goodSeeing (FWHM = 0.45 absent 0.45 =0.45 ′′ ) and badSeeing (FWHM = 0.70 absent 0.70 =0.70 ′′ ) tracts based on the statistics of the difference between output forced cModel color and input value ( Δ ​ Color Δ Color \Delta\mathrm{Color} ) shown in Fig 12 . Other details are the same with Table 2 .

Table: Summary of forced cModel Colors
$i_{\mathrm{Input}}$	Color	$\left(g-i\right)$		$\left(i-y\right)$
	FWHM	$0.45$′′	$0.70$′′	$0.45$′′	$0.70$′′
(mag)		(mag)	(mag)	(mag)	(mag)
20.0	${\sigma }_{\Delta \mathrm{Color}}$	0.068	0.051	0.072	0.046
		0.013	0.002	- 0.006	- 0.001
21.0	${\sigma }_{\Delta \mathrm{Color}}$	0.080	0.059	0.062	0.040
		0.010	0.003	- 0.003	- 0.004
22.0	${\sigma }_{\Delta \mathrm{Color}}$	0.089	0.072	0.071	0.059
		0.002	0.002	- 0.005	- 0.007
23.0	${\sigma }_{\Delta \mathrm{Color}}$	0.105	0.090	0.109	0.103
		- 0.018	0.001	- 0.002	- 0.005
24.0	${\sigma }_{\Delta \mathrm{Color}}$	0.136	0.127	0.175	0.170
		- 0.008	0.006	0.012	- 0.007
25.0	${\sigma }_{\Delta \mathrm{Color}}$	0.179	0.185	0.227	0.241
		- 0.003	0.019	0.053	0.006

Equations

$$b(\mathrm{A})\equiv 1-\frac{{\int }_{{\mathbb{R}}^2}\mathrm{d}x\mathrm{d}y{\mathcal{N}}_{\mathrm{A}}(\mathbf{x}|{\mu }_{\mathrm{A}},{\mathbf{\Sigma }}_{\mathrm{A}})F_{\mathrm{A}}(\mathbf{x})}{{\int }_{{\mathbb{R}}^2}\mathrm{d}x\mathrm{d}y{\mathcal{N}}_{\mathrm{A}}(\mathbf{x}|{\mu }_{\mathrm{A}},{\mathbf{\Sigma }}_{\mathrm{A}})F_{\mathrm{total}}(\mathbf{x})},$$

Full text

1 Introduction

Wide-field, multi-band imaging surveys have stepped onto the central stage of modern astrophysics and cosmology over the past decade. These efforts will soon be replaced with even more ambitious programs such as the Large Synoptic Survey Telescope (LSST)111https://www.lsst.org/, the Wide-Field Infrared Survey Telescope (WFIRST; Dressler et al. 2012; Spergel et al. 2015)222https://wfirst.gsfc.nasa.gov/, and the Euclid project (Laureijs et al. 2012)333http://sci.esa.int/euclid/. Among many ongoing efforts, the Subaru Strategic Program (SSP; Aihara et al. 2017a)444http://hsc.mtk.nao.ac.jp/ssp/, which uses the Hyper Suprime-Cam (HSC; Miyazaki et al. 2012) on the prime focus of the Subaru telescope, is the most efficient in terms of etendue555The collecting area multiplied by the field of view.. Surveys such as HSC, and other to follow, will provide stringent constraints on the cosmological model, characterize the evolution of galaxies, map out the stellar structure of our Milky Way, and are poised to discover a large number of interesting transient objects.

Before we can tackle outstanding scientific questions, we must first learn how to handle the the large amounts of data666Until Feb 2017, SSP has accumulated ${\sim}300$ TB of data products. generated by these projects while satisfying strict requirements for high quality image processing with accurate measurement for the magnitudes and shapes of stars and galaxies. Data handling becomes increasingly challenging in the age of modern imaging surveys am we aim to characterize and account for subtle effects related to charge-coupled devices (CCDs). Cameras are made up of multiple CCDs, each with slightly different characteristics, and have large fields-of-views (FoV) over focal planes that are not perfectly flat. During observations, the seeing and background conditions display spatial and temporal variations across the FoV. The full-depletion, thick CCDs selected for HSC enable long exposure times and have excellent red-sensitivity, but they also suffer from the so-called “brighter-fatter” effect (Antilogus et al. 2014; Guyonnet et al. 2015), which means that brighter stars have larger Point Spread Functions (PSFs) than fainter stars. These variations and effects make accurate astrometric and photometric calibration, background subtraction, and point spread function (PSF) modeling intrinsically difficult (e.g., Schlafly et al. 2012). Furthermore, as surveys reach to increasingly deeper detection limits (e.g., the SSP Wide layer reaches a $5\ \sigma$ point source detection limit of ${\sim}26.4$ mag in $i$-band), the number of objects per unit area increases. Hence, deep surveys become more sensitive and subject to the effects of blending. The challenge of modern imaging surveys call for photometric measurement methods that are more powerful and precise than those used in previous, shallower surveys.

These challenges are not merely technical details; their resolution is crucial to achieving key scientific goals. For instance, weak gravitational lensing (WL; Kaiser & Squires 1993; Bartelmann & Schneider 2001) is a powerful tool for measuring the large-scale distribution of dark matter. However, WL measurements critically depend on our ability to measure the shape and photometric redshifts of background galaxies with high precision. Photometric redshifts (e.g., Benítez 2000; Bolzonella et al. 2000; Ilbert et al. 2009) are fundamentally important for studying the evolution of galaxies, and the “drop-out” method is critical for selecting high-redshift galaxies (e.g., Steidel et al. 1996). However, both of those methods rely on the availability of accurate multi-band photometric measurements. With increasingly large surveys, and with more stringent requirements on data quality, quality control also becomes a pressing issue.

In this paper, we present a Python-based software package called SynPipe that injects synthetic objects into HSC images and which interfaces with the HSC data reduction pipeline (hscPipe). Among other applications, SynPipe can be used to perform quality control and to characterize the performance and limitations of hscPipe. Several tools with similar goals have been developed (e.g., Chang et al. 2015; Suchyta et al. 2016) for the Dark Energy Survey (DES; The Dark Energy Survey Collaboration 2005).

In Section 2, we briefly introduce the HSC survey and the current status of data reductionin. In Section 3, we explain SynPipe design and implementation. In Section 4, we demonstrate SynPipe usage via straightforward tests, we show the main results for the general photometric quality of synthetic stars and galaxies in Section 5. We then summarize the work and discuss future developments in Section 7.

The code for SynPipe, along with documentation and examples, is made available on GitHub at: https://github.com/dr-guangtou/synpipe.

2 Hyper-Suprime Cam Subaru Strategic Program (HSC Survey)

2.1 Status of the Survey

Taking advantage of the new prime focus camera on the 8.2–m Subaru telescope, the ambitious HSC survey consists of three layers: Wide, Deep, and UltraDeep. The Wide layer will map a total of ${\sim}1400$ deg2 of sky in five broad bands ($grizy$; Kawanomoto et al. in prep.). The Deep (four separated fields; ${\sim}27$ deg2) and UltraDeep (two separated fields; ${\sim}3.5$ deg2) layers use a few additional narrow-band filters and employ a slightly different surveying strategy. Aihara et al. (2017b) describes the HSC survey in more detail, and identifies the HSC collaborators.

The HSC camera (Miyazaki et al. 2012) is made up of 124 full-depletion thick CCDs: 112 for science and another 12 for guiding and focusing. The camera has a circular FoV with a 1.5 deg diameter. Each CCD contains 2048$\times$4096 pixels and the sizes of pixels are 0.168*′′*. For more details about the HSC camera, please see Miyazaki et al. (in prep.).

In February 2017, the HSC collaboration released the first 1.7 years of data to the public (DR1; Aihara et al. 2017a)777https://hsc-release.mtk.nao.ac.jp . For our work, we focused on data from the Wide layer as they relate to key scientific goals of the HSC survey, which include weak lensing cosmology, galaxy evolution, and studies of galaxy clusters. The Wide layer data in DR1 correspond to ${\sim}108$ deg2 spread over six fields (XMM–LSS, GAMA09H, WIDE12H, GAMA15H, HECTOMAP, and VVDS) 888A small AEGIS field is also observed for calibration purposes.. Except for the HECTOMAP field, the regions are all close to the equator. In the Wide layer, the $g$ and $r$ bands have 10 minute exposures broken into four dithers. The $i$, $z$, and $y$ bands have 20 minute exposures broken into six dithers. The survey prioritizes the observations so that the $i$-band has the best seeing conditions to improve galaxy shape measurements for weak lensing science. The $i$ band data in the Wide layer reach a $5\sigma$ point source limiting magnitude of $i{\sim}26.4$ mag and have a median seeing with FWHM${\sim}0.6$′′ in the $i$ band. Aihara et al. 2017a provides details on the first data release and data status.

2.2 The HSC Data Reduction Pipeline

The HSC data are reduced using a pipeline that builds on the prototype pipeline being designed for the Large Synoptic Survey Telescope’s Data Management system (Ivezic et al. 2008; Axelrod et al. 2010; Jurić et al. 2015) and is described in Bosch et al. (in prep.). DR1 data are reduced by hscPipe v4.0.5. BecauseSynPipe is intrinsically connected to the complex data reduction processes in hscPipe, we briefly introduce the main steps below along with a few key HSC/LSST terms.

Single-Visit Processing: Each individual exposure is called a Visit and is assigned an even integer. After initial data screening (considering the background level, seeing, and transparency Furusawa et al. in prep.), hscPipe subtracts overscan, bias, and dark frames, and performs flatfielding to the single CCD images. During this step, hscPipe also generates variance and mask images, subtracts the background, and provides initial astrometric and photometric calibrations. The photometric calibration is based on data from the Panoramic Survey Telescope and Rapid Response System (Pan-STARRS) 1 imaging survey (Schlafly et al. 2012; Tonry et al. 2012; Magnier et al. 2013). SynPipe uses the spatially varying PSF models, photometric zeropoints, and the World Coordinate System (WCS; Greisen & Calabretta 2002a; Calabretta & Greisen 2002b) corrected for optical distortion provided by hscPipe. 2. 2.

Multi-visit Processing: After single-visit processing, hscPipe warps and mosaics the reduced CCD images into much deeper coadd images while improving the astrometric and photometric calibrations via processes similar to the über-calibration in the Sloan Digital Sky Survey (SDSS) (Padmanabhan et al., 2008). SynPipe will use these improved calibrations. hscPipe organizes these coadd images into equiareal rectangular regions, or tracts, which are predefined as iso-latitude tessellations. One tract covers approximately $1.7\times 1.7$ degrees2 and adjacent tracts overlap each other by ${\sim}1$′. Each tract is further divided into $9\times 9$ Patches. A Patch is a $4200\times 4200$ pixels rectangular region, and adjacent Patches overlap each other by 100 pixels. 3. 3.

Multi-band Measurements: To achieve consistent photometry across all filters, hscPipe first detects and deblends objects on coadd images in each band independently (the unforced measurements). The collection of above-threshold pixels for each object is referred to as a footprint. hscPipe merges the footprints and flux peaks in the different bands and then selects a reference band for each object based on its $S/N$ in each band (usually this corresponds to the $i$-band). Centroids, shapes, and other non-amplitude parameters are then fixed to the values from the reference band. hscPipe then performs forced photometry using these fixed quantities. The goal of the forced photometry step is to generate accurate colors. 4. 4.

HSC Photometry: The multi-band catalogs generated by hscPipe contain various types of photometric measurements (see Aihara et al. 2017a for details). Here, we focus on characterizing the the forced psf and cModel photometry. The psf photometry should provide the most appropriate magnitudes and colors for point sources, while the cModel should be used for galaxies.

• •

hscPipe uses matched-filter method to derive PSF magnitude with the centroid fixed. The algorithm matches the image of a star with the PSF model multiplied by an amplitude. This amplitude parameter is estimated via the inner product of the image and PSF model after shifting the model to the centroid of the star, then divide by the effective area of the PSF model.

• •

The HSC cModel algorithm is described in Bosch et al. (in prep), and is based on the version for the SDSS cModel magnitude (Lupton et al. 2001; Abazajian et al. 2004). It fits objects using both a de Vaucouleurs and an exponential profiles convolved with the PSF model. cModel then linearly combines the two results to find the best fit to the galaxy image. cModel magnitudes are designed to yield accurate fluxes and colors for galaxies. Star–galaxy classification is also performed based on the difference between psf and cModel magnitudes.

DR1 data quality has been vetted in Aihara et al. (2017a) by cross-checking with the Pan-STARRS 1 imaging survey to examine the behavior of the stellar sequence. Generally speaking, the PSF photometry is accurate at the 1–2% level for bright stars, and the astrometry is accurate to ${\sim}10$ and 40 max internally and externally.

3 SynPipe Overview

3.1 Design

hscPipe is a complex data reduction pipeline that still faces many challenges and is under active development. Unlike most publicly available photometric pipelines, it involves high-level reduction processes that produce not only deep coadd images but also a series of science-ready catalogs. To perform tests of the data products that result from hscPipe, SynPipe is designed to satisfies a few basic standards:

• •

Realistic Images: Instead of using a fully generative approach to simulate full HSC-like images from the ground up (e.g., Chang et al. 2015), SynPipe injects synthetic objects into real HSC images so that all the realistic features of the real data (e.g., blended objects; proximity to bright objects, bleeding trails, other optical artifacts) can be included in the test. The impact of these features can be important for studies that care about completeness and the selection function. In SynPipe, point sources are simulated using the HSC PSF model, and a realistic galaxy model is created by the modular galaxy image simulation toolkit, GALSIM (Rowe et al. 2015) 999https://github.com/GalSim-developers/GalSim.

• •

Authentic Data Processing: To follow the data reduction process as realistically as possible, we choose to start from the single-visit images instead of directly putting synthetic objects on the final coadd images (e.g., Suchyta et al. 2016). Through this approach, subtle effects like seeing differences among visits and small errors in astrometric calibration can be taken into account. Every synthetic object will experience all the steps we would use for a real object: detection, deblending, stacking, and measurement. This can be important for challenging tasks like WL measurements or the detection of high-$z$ galaxies. The downside of this choice is that it slows down the overall run-time because we need to inject synthetic objects into all the visits that contribute to a tract.

• •

Flexible Capabilities: SynPipe is designed to be flexible enough to be useful for HSC users with a range of scientific goals. We provide default catalogs to generate samples of synthetic stars and galaxies with realistic magnitudes and color distributions along with tools to help the user work on HSC DR1 data. However, the users can also supply their own input catalogs best suited for their applications. SynPipe is already being used in a wide range of scientific topics (see Section 7 for details).

3.2 SynPipe Implementation

In this section, we describe the SynPipe test implementation illustrated in Fig 1.

3.2.1 Preparation

To create a catalog of synthetic objects, the user first needs to select which HSC data and which input synthetic galaxy catalog to use.

• •

Information about the data. This corresponds to the location of HSC images and a list of visits to be used. SynPipe can also help the user identify all the visits that contribute to one tract in any given band. SynPipe also provides tools to create an optional bad-pixel mask (e.g., bright object, bleeding trails) for a specific tract so that the user can avoid putting synthetic objects on problematic regions.

• •

Input catalog. This is a catalog in FITS (Flexible Image Transport System) format that contains the coordinates, magnitudes, and model parameters of synthetic objects. The positions of synthetic galaxies can be specified via the input catalog, they can be distributed randomly over a given region, or on an evenly-spaced grid (the grid option is useful in that it avoids blends between Synthetic objects, e.g. see Murata et al. in prep.).

Synthetic objects can be one of the following:

• •

Point source: SynPipe simply uses the PSF model from hscPipe as the model for a point source (stars and/or quasi-stellar objects). hscPipe uses a special version of PSFEx (Bertin 2011, 2013) to characterize PSF as a function of position and the PSF model can be reconstructed for any location on the image. To inject point sources, the user only needs to specify a catalog of coordinates and magnitudes.

• •

Sérsic models: SynPipe uses GALSIM v1.4 (Rowe et al. 2015) to simulate galaxies. GALSIM is an image simulation tool that was designed for the GRavitational lEnsing Accuracy Testing 3 (GREAT3) challenge (Mandelbaum et al. 2014). Currently, SynPipe allows synthetic galaxies to be modeled by a single or a double Sérsic (1963) profile101010More flexible model choices will be added later.. The Sérsic  profile is flexible enough to describe the overall flux distributions of galaxies near and far, both early-type or late-type. A double-Sérsic  model can simulate a galaxy with even more realistic structural details (e.g., bulge$+$disk). For each Sérsic  component, the user needs to provide the magnitude, effective radius ($R_{\mathrm{e}}$, in unit of arcsec), Sérsic  index ($n_{\mathrm{Ser}}$), axis ratio ($\mathrm{b}/\mathrm{a}$), and position angle (PA). External shear can also be applied to a synthetic galaxy ($g_{1}$ and $g_{2}$, see Rowe et al. 2015 for more details).

• •

Galaxies from the GREAT3 challenge: SynPipe also allows users to choose from parametric models or real high-resolution galaxy images used in the GREAT3 WL challenge. Real galaxy images are drawn from the HST/ACS F814W ($I$-band) images of galaxies in the COSMOS field (e.g., Scoville et al. 2007; Leauthaud et al. 2007). Instead of using real galaxy images, the users may also use models of COSMOS galaxies to $I_{\mathrm{Auto}}\leq 25.2$ mag via a method developed by Lackner & Gunn (2012). Parametric models of COSMOS galaxies include a single-Sérsic  and the Sérsic  bulge$+$Exponential disk model. The parameter values for these models are stored in the GalSim.COSMOSCatalog() catalog. To access them, the user can provide the ID number of COSMOS galaxies in the input catalog, or can simply opt to let SynPipe randomly select galaxies from the COSMOS input catalog.

3.2.2 Injection of Synthetic Objects into Single-visit Images

• •

With the input catalog and test data information in hand, SynPipe injects synthetic objects into single-exposure images (step addFakes.py; see the middle panel of Fig 1). SynPipe goes through individual CCD images that belong to each visit and decides which synthetic objects from the input catalog need to be injected. SynPipe uses the initial astrometric calibration of each single visit to convert the input coordinates into locations on the CCD image. For each synthetic object, SynPipe uses the reconstructed PSF for each exposure. The photometric zero point from the single-visit calibration is used to convert input magnitudes of synthetic objects into fluxes.

• •

With the help of GALSIM module, SynPipe simulate the images of synthetic objects.

• –

For point sources, SynPipe generates a rectangular cutout image of the PSF model with appropriate size and correct total flux.

• –

For galaxies described by single- or double-Sérsic  models, SynPipe passes the input Sérsic  index and effective radius to the GALSIM.Sersic function to create a Sérsic  component. After stretching and rotating the galaxy to the expected axis ratio and position angle via the shear and rotate methods, SynPipe assigns the correct flux to this component.

• –

For a double-Sérsic  model, SynPipe uses the GALSIM.Add method to combine two Sérsic  components.

• –

For parametric models from the GREAT3 catalog, SynPipe uses the COSMOSCatalog.makeGalaxy method to generate models.

• –

To inject real HST galaxy images, SynPipe calls the GALSIM .RealGalaxy method.

If necessary, an additional shear ($g_{1}$ and $g_{2}$) can be applied to the model at this point. Then, SynPipe passes the reconstructed PSF image into a GALSIMInterpolatedImage object and convolves it with the galaxy model using GALSIM.Convolve. After this, SynPipe uses the GALSIM.drawImage method to redner the image of the simulated galaxy. A component with a high Sérsic  index ($n_{\mathrm{Ser}}>4$) often requires a large image size to cover all of its flux, so SynPipe allows the user to truncate the model at a given radius (N$\times R_{\mathrm{e}}$). SynPipe will also give warning information when a component with $n_{\mathrm{Ser}}>6$ is encountered because it takes much longer to achieve accurate PSF convolution in such a model; and the result is, therefore, not a realistic galaxy model.

• •

After SynPipe creates the image of the simulated galaxy, it then shifts the image according to is location on the CCD on a pixel-by-pixel basis; SynPipe also crops the images when necessary. When that task is complete, SynPipe adds appropriate noise to the image based on the pixels it covers and the calibration information of the detector, and creates a variance image that reflects the influence of the synthetic object. Since each HSC CCD consists of 4 amplifiers, each with different characteristics, SynPipe carefully creates a corresponding gain map to provide accurate noise level. The added noise only accounts for the additional flux in the synthetic object, and will thus be subdominant compared to the noise already in the image for faint objects.

• •

In the final step, SynPipe adds the noise-added image and the variance map to the original CCD data while also adding a new FAKE mask bit to the mask plane. The middle panel of Fig 1 depicts the result of the addFakes.py step by showing an example CCD image before and after the synthetic objects are injected.

3.2.3 Stacking and Multi-band Measurements

The newly generated single-visit images have the same format and data structure as real HSC data. To perform stacking and measurements, SynPipe calls standard hscPipe routines.

The stack.py step takes the improved astrometric and photometric calibrations for each visit from the original reduction and creates coadd images that contain synthetic objects. The multiBand.py step then processes these coadd images and provides standard multi-band measurements in FITS catalogs that are grouped by Patch ID.

Using the infrastructure provided by hscPipe, the user can easily perform these steps in parallel, and the overall efficiency of SynPipe is similar to the real data reduction process. The data and catalog volumes is also similar to the original data reduction process.

The final step is to identify Synthetic galaxies in the output catalogs. SynPipe can match the input catalog to the output catalog (this contains a mix of real and synthetic galaxies) using a matching radius specified by the user. Each unmatched object from the input catalog is also passed to the results catalog with a unique label. In the case of multiple matches, the user can choose between returning only the closest match or returning all objects within the matching radius (the nearest one is labeled as such).

3.3 Limitations and Caveats

The limitations of the current SynPipe include the following:

Aihara et al. (2017a) points out that hscPipe tends to over-subtract the background around bright objects. Unfortunately, SynPipe now takes the original background subtraction on the single-visit image for granted; hence, it lacks the capability to test or help improve this problem. 2. 2.

SynPipe simply adopts the PSF measured by hscPipe and uses it as a model of point source and PSF convolution for a galaxy. Hence, it cannot be used to test how the uncertainty of the PSF modeling affects the photometry and shape measurements. This could be important for regions with exquisite seeing (e.g. FWHM${\sim}0.4$′′) that makes the PSF modeling difficult, and for accurate WL measurements (see Aihara et al. 2017a). 3. 3.

SynPipe works in a “unit” of visit for a single-exposure test, and uses a tract to test coadd images. In the case of tests that focus on specific regions that are much smaller than the size of a visit or a tract (e.g., quality of photometry around rich galaxy clusters), using SynPipe often leads to low efficiency.

4 Generation of Synthetic Dataset

We now describe the generation of the synthetic data set that we use to characterize the performance of hscPipe.

4.1 Data

We use tract$=9699$ in the VVDS field (median FWHM$=0.449$′′ in $i$ band; in this paper referred to as goodSeeing) and tract$=8764$ in the XMM-LSS field (median FWHM$=0.700$′′; badSeeing). tract$=9699$ also has better seeing in both $r$ and $z$ bands than the median conditions in those bands. The seeing conditions in $g$ and $y$ bands are very similar for both tract$=9699$ and tract$=8764$. For both tracts, the coadd image in each band includes data from 20–40 visits111111The large number of visits here is due to large dither pattern. Please see Fig 1.. These two tract are selected because they are not at the edge of a field, do not contain any extremely bright ($i<12$ mag) saturated stars, and are representative of both ”good” and ”bad” seeing121212Here, “bad” seeing only suggests that it is worse than the median seeing condition by HSC standard. conditions in the $i$ band.

4.2 Input Models for Stars and Galaxies

We use a synthetic star sample built using data from the HST/ACS catalog of Leauthaud et al. (2007). We first select stars from the Leauthaud et al. (2007) catalog (this classification is reliable down to $I_{\mathrm{F814W}}{\sim}25.2$ mag). We then match this sample against our HSC UltraDeep–COSMOS data (which reaches ${\sim}27.2$ mag in $i$ band). After applying basic quality cuts (see A), this procedure provides us with five-band HSC PSF photometry for 14,472 stars. To increase the sample size, we re-sample the five-band magnitude distributions using the astroML (VanderPlas et al. 2014) implementation of the extreme-deconvolution algorithm developed by Bovy Jo et al. (2011) 131313https://github.com/jobovy/extreme-deconvolution/ to generate 100,000 synthetic stars. Fig 2 shows the magnitude and color distributions of real COSMOS stars compared to the resampled distributions in all five-bands and in color-color space.

For synthetic galaxies, we use single-Sérsic  galaxy models of COSMOS galaxies with $I_{F814W}\leq 25.2$ mag. These are models from Lackner & Gunn (2012) applied to HST/ACS images and are included in GALSIM. Each model is described by a total of five parameters: magnitude, effective radius, Sérsic  index, axis ratio, and position angle. Although the single-Sérsic  model is not always the best choice for describing galaxies, its flexibility enables it to reasonably describe the flux distribution of most galaxies. Also, its simplicity makes it easy for us to diagnose potential problems with the cModel photometry. We exclude a tiny fraction (${\sim}4$%) of ill-behaved models that have a very high Sérsic  index ($n_{\mathrm{Ser}}>6.0$) or a very low central surface brightness ($\mu_{i}<24.0$mag arcsec*-2*). We further match this COSMOS sample with the HSC UltraDeep–COSMOS data. After removing objects with problematic photometry (see Appendix A for details), we obtain a sample of 58,210 synthetic galaxies with realistic five-band HSC cModel photometry. As shown in Fig 3, the majority of these galaxies are faint ($i<24.0$ mag) and barely resolved ($R_{\mathrm{e}}<1.0$′′). This sample is appropriate to test hscPipe’s general photometric behaviors, but the sample lacks relatively bright galaxies ($i<20.5$ mag).

We spatially distribute synthetic stars and galaxies randomly in our two selected tracts. For stars, we use a number density of 1,000 per CCD. For galaxies, we use a number density of 500 per CCD. Given the magnitude distributions of these objects, these numbers are high enough to ensure large sample of useful synthetic objects for the test, and will also not create unrealistic crowded images.

4.3 Generating SynPipe Data

We use HSC DR1 data stored at Kavli Institute for the Physics and Mathematics of the Universe (Kavli IPMU) for these tests. Using 108 cores, the addFakes.py step takes ${\sim}1.5$ hours per tract in each band for stars. The same process takes longer for galaxy tests (${\sim}3.0$ hours per tract in each band) because the GALSIM simulation is more time consuming. The stack.py step and multiBand.py step together take ${\sim}3.5$ hours per tract.

We match the results with the input catalogs using a 2–pixel matching radius to generate output catalogs for forced and unforced photometry in each band. For our tests, we reject synthetic objects locate within 2 pixels of a real HSC object. These “ambiguously blended” (e.g. Dawson et al. 2016) objects are extreme blends in which multiple objects are detected as a single object, and are not useful for photometric tests.

The detailed log of our runs can be found at goo.gl/VINOVP. The user should be able to reproduce the results presented in this work using this information. It can also serve as a brief manual for generating SynPipe data.

5 Photometric Performance Results

Here we discuss hscPipe’s general performance, assessed mainly through forced PSF photometry (for stars) and (cModel) photometry (for galaxies). Although hscPipe does provide other options (e.g., aperture and Kron photometry), these two are the only options that consider the effects of the PSF in different bands and are consistent across all bands in terms of position and shape.

5.1 PSF photometry of stars

In hscPipe, the PSF magnitude is derived using a matched-filter method that depends on the best-fit PSF model and uses third-order Lanczos interpolation to shift the PSF model. The error of the PSF magnitude from hscPipe only considers the per-pixel noise, and does not include centroid uncertainties. hscPipe also estimates aperture correction for PSF photometry. For more details please see Bosch et al. (in prep.).

We randomly inject ${\sim}100,000$ stars into each tract in all five bands. Typically, there are ${\sim}80000$ stars with $i_{\mathrm{PSF}}<26.0$ in one HSC tract. Approximately ${\sim}3$–$4$% of the stars are located within 2 pixels of the centroids of real objects; we remove those stars from the sample to avoid confusion in the comparisons. For matched stars, we select the primary detections (detect.is-primary=True) that have good photometric quality (see Appendix A for details). This gives us ${\sim}83,000$ stars in each tract.

Next, we exclude stars that are misidentified by hscPipe as extended objects using the classification.extendedness parameter in each band. The fraction of misclassification is ${\sim}10$–$20$%, and clearly depends on seeing conditions. For the same reason, the $g$-band has the highest fraction of misclassification, while $i$-band is recommended for selecting point sources. The star–galaxy separation issue is discussed more in Section 6.2.

In the following comparisons, we show the results from the goodSeeing and badSeeing tracts separately because it is important to understand how the seeing conditions affects photometric accuracy as well as to test whether hscPipe can deliver unbiased photometry under different seeing conditions.

Besides seeing, the degree to which an object is blended with other objects is another factor that influences photometric accuracy. To quantify the impact of blending effects, we use the “blendedness” parameter, $b$. This parameter describes how much any given object overlaps with neighboring objects (see Appendix B and Murata et al. in prep.). We divide our sample according to the blendedness parameter. A value of $b=0$ corresponds to isolated objects whereas objects with $b=1$ complete overlap with other objects. Here we use $b>0.05$ to define highly blended stars (${\sim}4$–$9$% are blended according to this criterion). The fraction of highly blended stars slightly increases in the badSeeing tract. The impact of blendedness will be discussed more in Section 6.3.

5.1.1 Relationship between stellar magnitude and $\mathrm{S}/\mathrm{N}$

The $\mathrm{S}/\mathrm{N}$ for PSF photometry is defined as $\mathrm{Flux}_{\mathrm{PSF}}/\mathrm{Flux\_Err}_{\mathrm{PSF}}$ in each band. Figure 4 shows the relationship between stellar magnitude and $\mathrm{S}/\mathrm{N}$ in the $g$- and $i$-bands. This figure shows the expected $\mathrm{S}/\mathrm{N}$ of stars as a function of magnitude in the Wide layer. The slopes and scatters of these relations are very similar to the ones using real stars on these two tracts.

Fig 4 show that seeing conditions impact the $\mathrm{S}/\mathrm{N}$ of point sources at fixed input magnitude. In $i$-band, the badSeeing (0.70*′′) tract shows systematically lower $\mathrm{S}/\mathrm{N}$ than the goodSeeing (0.45′′*) tract. The $\mathrm{S}/\mathrm{N}$ is similar for the two $g$-band tracts because they share similar seeing conditions.

At \mathrm{S}/\mathrm{N}$$=5, HSC Wide can detect stars as faint as ${\sim}26.5$ mag in both $g$ and $i$ bands, which is consistent with the values found in Aihara et al. (2017a) for average seeing conditions. However, it is worth reminding HSC data users that the detection limit will exhibit spatial variations due to the seeing conditions.

5.1.2 Precision and Accuracy of PSF magnitudes

We now investigate the performance of the PSF photometry in each of the five bands independently. Fig 5 shows the difference between the input magnitude versus the hscPipe forced PSF magnitude ($\Delta\mathrm{mag}_{\mathrm{PSF}}$) as a function of input magnitude. We separate these stars into seven input magnitude bins. In each bin, we characterize the statistical precision of the PSF magnitude using the standard deviation of the distribution ($\sigma_{\Delta\mathrm{Mag}}$). Meanwhile, we characterize the statistical accuracy of the PSF magnitude via the mean magnitude difference in each bin ($\left<{\Delta\mathrm{Mag}}\right>$). This also informs us whether the PSF photometry is biased at certain input magnitude.

The overall performance of the hscPipe forced PSF magnitude is excellent. The median $i$-band PSf magnitude precision for the goodSeeing tract is ${\sim}0.014$ mag (${\sim}1.3$%) at $i_{\mathrm{Input}}{\sim}19.0$ mag. At $i_{\mathrm{Input}}{\sim}24.0$ mag, the precision of the PSF magnitude is at ${\sim}3$% level (${\sim}0.030$ mag statistical scatter). At $i_{\mathrm{Input}}{\sim}25.0$ mag, the precision decreases to ${\sim}6$% with an average ${\sim}0.062$ mag difference.

The precision of PSF magnitude for the badSeeing tract shows similar performance at $i_{\mathrm{Input}}<23.5$ mag. At fainter magnitudes, the precision slightly degrades. The badSeeing tract has ${\sim}4$% precision at $i_{\mathrm{Input}}{\sim}24.0$ mag and ${\sim}11$% precision at $i_{\mathrm{Input}}{\sim}25.0$ mag. Aihara et al. (2017a) evaluates the precision of PSF magnitude via external comparisons with the PS1, PV2, and SDSS data at $i<21$ mag, and finds it at 1-2% level, which is consistent with our results. However, at fainter magnitudes, external comparisons become difficult due to the lack of imaging matched to HSC depths. The results from SynPipe hence provide useful evaluations of the precision of the HSC photometry down to our detection limit141414Strictly speaking, the precision reported here should be considered as upper limits because SynPipe currently does not consider the systematic uncertainties in PSF modeling. But this effect should be subdominant.

Fig 5 also shows that the precision of the PSF photometry is not filter dependent. The precision of forced PSF magnitudes in $r$ and $z$ is similar to the results for $i$-band. For $r$ band we find 1.5-4.0% precision at $r_{\mathrm{Input}}<24.0$ mag and ${\sim}8$% precision at $r_{\mathrm{Input}}{\sim}25.0$. For $z$-band we find 1.0-5.0% precision at $z_{\mathrm{Input}}<24.0$ mag and ${\sim}10$% precision down to $z_{\mathrm{Input}}{\sim}25.0$ mag).

The precision for $g$ and $y$ bands becomes slightly worse at the very faint end. For $g$ band, the precision is 1.5-7.0% at $g_{\mathrm{Input}}<24.0$ mag and is ${\sim}13$% down to $25.0$ mag. For $y$ band, we find 1.5-5.0% precision at $y_{\mathrm{Input}}<23.0$ mag and ${\sim}12$% down to $24.0$ mag. These differences are however consistent with the differences in the seeing conditions between the filters

The PSF photometry for relatively isolated stars is accurate and unbiased in $i$ and $z$ bands down to faint magnitudes. For the $g$, $r$, and $y$ bands, we find mean $\Delta\mathrm{mag}_{\mathrm{PSF}}$ that are close to the PSF flux errors estimated by hscPipe. hscPipe tends to underestimate the fluxes of stars in these bands by ${\sim}0.01-0.02$ mag at $<24.0$ mag. The exact cause of this offset is unclear but the levels of bias is quite small and not a major concern.

However, Fig 5 also shows that blending has a strong impact on photometry. On average, hscPipe systematically underestimates the total fluxes of stars that are subject to blending effects by $0.05$–$0.10$ mag at a fixed input magnitude. We see the same effect in all bands, and the impact of blendedness becomes increasingly significant at fainter magnitudes. It is important to bear this caveat in mind when using PSF photometry for point sources in HSC data.

Finally, we also perform similar tests for the unforced PSF photometry in all five bands, and find similar results. To measure the forced PSF photometry, hscPipe fixes the centroid of the PSF model across all five bands. A difference between forced and unforced PSF magnitudes would therefore be an indication of photometric uncertainties arising from inaccurate astrometric calibrations and PSF modeling across different bands151515On average, the centroids are more accurate in forced photometry as they are defined using the band with higher $\mathrm{S}/\mathrm{N}$. Fig 6 shows the difference between forced and unforced PSF photometry for the $g$ and the $y$ band. The overall differences are small ($<0.01$ mag). At very faint end ($g_{\mathrm{Input}}>24.0$ mag and $y_{\mathrm{Input}}>23.5$, the unforced photometry slightly overestimate the fluxes of stars comparing to the forced photometry.

The precisions and accuracies of forced PSF magnitudes shown here are summarized in Table 1

5.1.3 Precision and Accuracy of PSF colors

PSF photometry is the most appropriate way to measure colors for point sources; therefore, the accuracy of the PSF color estimates is important to many scientific goals of the HSC survey (e.g., study of the Milky Way structure, selection of unique stellar objects or high-redshift quasars, and accurate star–galaxy separation).

In Fig 7, we evaluate the precision and accuracy of the forced $(g-i)$ and $(i-y)$ PSF colors by comparing the differences between input and output colors ($\Delta\mathrm{Color}$) with both the input magnitudes and colors. Fig 7 shows that hscPipe provides precise and accurate PSF colors for synthetic stars with realistic color distributions down to very faint magnitudes. The average precision for $(g-i)$ is ${\sim}0.023$ mag at $i_{\mathrm{Input}}{\sim}19.0$. The statistical scatter increases to ${\sim}0.16$ mag at $i_{\mathrm{Input}}{\sim}25.0$ mag. The $(i-y)$ color displays similar precision. The forced PSF color measurements are not biased through the entire ranges of input magnitudes and colors.

Fig 7 shows that the precision of PSF colors do not depend the seeing conditions. For instance, the precision of $(g-i)$ for the badSeeing tract is only slightly worse at the very faint end (${\sim}0.18$ mag at $i_{\mathrm{Input}}{\sim}25.0$ mag) compared to the goodSeeing tract.

Fig 8 displays the precision of PSF colors by comparing the color distributions of the input sample to the recovered color distributions. Using four different colors and two color–color planes, we show that the forced PSF colors accurately recover the distributions of all four colors and the color-color distributions.

As for highly blended stars, Fig 7 also shows that highly blended stars have worse precision and accuracy in their colors compared to isolated stars. At fixed input magnitude, the precision in $(g-i)$ is ${\sim}0.1$ mag worse for highly blended stars. Also, for blended stars, $(g-i)$ shows a bias towards bluer colors, while $(i-y)$ shows a bias towards redder colors. However, such biases appear to be less severe in terms of colors than in terms of PSF magnitudes.

The statistical uncertainties of forced PSF colors are summarized in Table 2

5.2 cModel photometry of Galaxies

For galaxies, hscPipe uses a cModel photometry algorithm that is an improved version of the SDSS cModel photometry. For details about the cModel algorithm, please see description in Section 2.2 and Bosch et al. (in prep.). Despite the limitations of the cModel method (e.g., sensitivity to the background subtraction and to deblending failures), it can deliver robust PSF-corrected fluxes and colors for galaxies.

Typical tract in the Wide layer contains ${\sim}400000$ extended objects with $i_{\mathrm{CModel}}<25.5$ mag. We randomly inject ${\sim}40,000$ synthetic galaxies (additional ${\sim}10$%) into each tract, and will not create over-crowded situation. Using similar approach for synthetic stars, we select galaxy samples for photometric comparisons. Synthetic galaxies that are located within 2 pixels of the centroids of real objects are removed from the sample (they represent ${\sim}3$–$7$% of the input sample). After imposing quality cuts on the recovered synthetic galaxy sample (see Appendix A for details), we have ${\sim}30,000$ galaxies in each tract.

We find that hscPipe mis-classifies lower than $1$% of these objects as point sources in the goodSeeing tract. But the fraction of misclassified galaxies increases to $4$% in the goodSeeing tract. We will discuss star/galaxy separation further in Section 6.2.

We define a sample of highly blended galaxies by imposing the cut $b>0.05$161616We already removed the ambiguously blended objects. These objects have high blendedness, but are still distinctive.. Due to the extended nature of galaxies, the $b$ distribution of synthetic galaxies is more skewed toward high values (especially for the badSeeing tract) compared to the b distribution of stars. About $7$–$9$% of synthetic galaxies are highly blended with $b>0.05$. The fraction is slightly higher in the badSeeing tract, and therefore the $g$ band also has higher fraction of highly blended galaxies. In our plots, we will highlight highly blended galaxies and will discuss blending effects further in Section 6.3.

5.2.1 Input magnitude and the $\mathrm{S}/\mathrm{N}$ of cModel photometry

Fig 9 shows the relation between the input magnitudes and the $\mathrm{S}/\mathrm{N}$ of synthetic galaxies. The $\mathrm{S}/\mathrm{N}$ is the ratio of the cModel flux and the flux error measured by hscPipe. Here the adopted $\mathrm{S}/\mathrm{N}$ corresponds to the average $\mathrm{S}/\mathrm{N}$ over entire footprint. The center of the galaxy typically has high $\mathrm{S}/\mathrm{N}$ than this value.

Fig 9 shows that the cModel photometry displays a well-behaved relation between input magnitude and output $\mathrm{S}/\mathrm{N}$. Compared to stars, the $\mathrm{S}/\mathrm{N}$ from cModel for synthetic galaxies shows a larger scatter at fixed input magnitude. This is likely due to the fact that galaxies have a wider range in sizes and shapes than point sources. In $i$ band, a typical $25.2$ mag galaxy has \mathrm{S}/\mathrm{N}$${\sim}20 from cModel, but there are also galaxies at the same magnitude which have $\mathrm{S}/\mathrm{N}$ below 5. The HSC Wide layer can detect galaxies as faint as $i{\sim}26.0$ mag but the sample becomes incomplete at these faint magnitudes. This also introduces a selection bias if galaxies with certain structural properties (e.g., more extended and lower surface brightness) are harder to detect than others. HSC survey data users who want to select flux-limited galaxy samples, or who intend to study the population of faint (e.g., high-$z$) galaxies should keep this in mind. Finally, we also find that a worse seeing leads to a lower $\mathrm{S}/\mathrm{N}$ at fixed magnitude (see Fig 4 for $i$-band).

5.2.2 Precision and Accuracy of the cModel magnitude

Fig 10 shows the precision ($\sigma_{\Delta\mathrm{Mag}}$) and accuracy ($\left<{\Delta\mathrm{Mag}}\right>$) of cModel magnitude in each band using the same format as Fig 5.

The overall performance of cModel photometry is reasonable down to $i_{\mathrm{Input}}=25.2$ mag. Compared to the PSF photometry for stars, the statistical uncertainties of cModel magnitudes are larger for galaxies, which is expected given the diversity of galaxy shapes and sizes that adds complexity to the model-fitting process. At the same time, Fig 10 shows that the cModel algorithm in hscPipe provides unbiased and consistent photometry for galaxies across different bands and seeing conditions.

The typical precision of $i$-band flux using forced cModel is at ${\sim}10-14$% level in the goodSeeing tract at $i_{\mathrm{Input}}<24.0$ mag. It moderately degrades to ${\sim}18$% at $25.0$ mag. The performance is similar in $i$ band for the badSeeing tract. At $20.0<i_{\mathrm{Input}}<24.0$ mag, the accuracies of forced cModel magnitudes change between

$0.017$ mag to

$0.023$ mag. At $24.0<i_{\mathrm{Input}}<25.5$ mag, $\left<{\Delta\mathrm{Mag}}\right>$ is around $\raisebox{0.4pt}{\scalebox{0.8}{-}}0.006$ mag, which suggest that forced cModel photometry is unbiased down to the very faint end.

It is worth noting that the flux uncertainty from SynPipe are significantly larger than the flux errors from hscPipe, which does not take into account the systematic uncertainties involved in the modeling fitting process. Even though our synthetic galaxy sample is comprised of objects with simple single-Sérsic  models, cModel algorithm can only approximate them to a certain accuracy because of the background noise, deblending uncertainties, and priors imposed on model parameters. Given that real galaxies are more complex in structure, our quoted uncertainties should be treated as lower limits.

The precision of forced cModel photometry is consistent across all filters. For the $g$, $r$, and $z$ bands, the statistical uncertainties are very similar to $i$-band results between $20.0$ and $25.0$ mag. The forced cModel magnitudes are also unbiased for these three filters across the entire input magnitude range. The precision for the $y$ band is slightly worse, ranging from 10% to 17% in the same magnitude bins. In addition, the forced cModel tends to overestimate the total flux in the $y$ band at $>24.0$ mag. This bias in the $y$ band flux is equal to

0.07 at $24.0$ mag to

0.22 mag at 25.0 mag. Worse seeing conditions and a higher background noise level mean that it is more difficult to detect faint objects in the $y$ band. At $y_{\mathrm{Input}}{\sim}24.0$, the average $\mathrm{S}/\mathrm{N}$ for cModel is already around the detection threshold.

Our synthetic galaxy sample is dominated by faint galaxies ($i_{\mathrm{Input}}>24.0$ mag) that are crucial to key scientific goals of the HSC survey. But it also means poor statistics for bright galaxies. We do find that at $i_{\mathrm{Input}}<20.0$, the forced cModel photometry starts to systematically underestimate total fluxes. Based on external comparisons, the behavior of cModel for bright galaxies may depend on galaxy types (e.g., Aihara et al. 2017a). Also, at bright end, another known issue is that the current hscPipe over-deblends around bright objects. Inappropriately weight priors in cModel also leads to poorer photometric quality for bright galaxies. Finally, at the imaging depth of HSC ($>29.0$mag arcsec*-2* in the Wide Layer $i$ band), cModel may simply not be a good choice for modeling bright galaxies (e.g., massive elliptical galaxies; see Huang et al. in prep. for more details).

We also find that highly blended galaxies tend to have systematically underestimated total fluxes (with an average offset ${\sim}0.1$ mag) and higher statistical uncertainties in all five bands. However, the contrast between relatively isolated and highly blended galaxies in photometric performance is not as stark as for synthetic stars (Fig 5).

We also test the unforced cModel photometry in all five bands, and the results suggest similar precision. Fig 11 compares the difference between forced and unforced cModel in $g$ and $y$ bands. The main noticeable trend is that galaxies brighter than $24.0$ mag in the $g$ band have systematically brighter forced cModel magnitudes, which suggests that the forced cModel is more accurate in the $g$ band as the model parameters are determined using band with higher $\mathrm{S}/\mathrm{N}$.

The precisions and accuracies of forced cModel magnitudes shown here are summarized in Table 3

5.2.3 Precision and Accuracy of the cModel color

Precise and unbiased five-band cModel colors from hscPipe are crucial for photometric redshift estimates and spectral energy distribution (SED) fitting results. They are also key, for example, to the selection of high-$z$ Lyman-break galaxies (LBG).

Fig 12 evaluates the precision and accuracy of forced $(g-i)$ and $(i-y)$ cModel colors using the same format as Fig 7. We find that hscPipe provides reliable cModel colors for synthetic galaxies down to $i_{\mathrm{Input}}{\sim}25.0$ mag. For $(g-i)$ colors, the statistical uncertainty at $i_{\mathrm{Input}}{\sim}20.0$ mag is ${\sim}0.07$ mag. The statistical uncertainty increases to ${\sim}0.18$ mag at $i_{\mathrm{Input}}{\sim}25.0$ mag. The $(i-y)$ colors show similar results, except the precision at $i_{\mathrm{Input}}>24.0$ mag becomes slightly worse.

The performance of forced cModel color does not strongly depend on seeing and is unbiased for the entire range of magnitudes and colors that we have tested. The only noticeable feature is that the $(i-y)$ color at $i_{\mathrm{Input}}>24.0$ is systematically redder than the real values by $\raisebox{0.4pt}{\scalebox{0.6}{+}}0.05$ mag. As mentioned above, this is not surprising given the shallower $y$ band imaging depth. Highly blended galaxies are less precise in their forced cModel colors compared to isolated galaxies. Highly blended galaxies on average show slightly bluer $(g-i)$ colors and redder $(i-y)$ colors compared to their input colors.

Fig 13 compare the input color–color distributions with the recovered ones. The general 2–D distributions are well recovered but the precision is not as good as for stars, especially in the $(i-z)$ vs. $(z-y)$ plane. This is expected given the narrow dynamical ranges of these colors in the redder filters. The forced cModel tends to slightly underestimate the $(g-r)$ and $(i-z)$ colors at the very “blue” end, while overestimating the $(i-y)$ color at the very red end.

We note that our synthetic galaxies do not have color gradients. In reality, color gradients will complicate the situation and hence the statistical uncertainties shown here should be considered as lower limits.

The statistical uncertainties of forced cModel colors are summarized in Table 4

6 Other Performance Tests

In this section, we turn our attention to astrometric calibration, star-galaxy separation, blending effects, and galaxy shape measurements.

6.1 Astrometric Calibration

We inject synthetic objects into single-visit images using the initial (and less accurate) astrometric calibration. During the image stacking step, the joint calibration process improves the astrometric solutions. Therefore, differences between the input and output coordinates from SynPipe can help us understand how much has the joint calibration step changes the astrometric calibration, and how well can hscPipe measure the central coordinates of stars and galaxies.

Fig 14 shows the distribution of astrometric offsets for synthetic stars and galaxies in goodSeeing and badSeeing tracts. We find small systematic offsets at the ${\sim}10$–20 mas level, which is similar to the uncertainties of astrometric calibration quoted in Aihara et al. (2017a). The two tracts show different systematic offsets, while the synthetic stars and galaxies show coherent offsets in the same tract. This shows that the initial astrometric calibration of HSC has very reasonable accuracy. It also suggests that hscPipe can find the centroids of stars and galaxies with precisions that are consistent with the astrometric calibration uncertainty under moderately different seeing conditions.

6.2 Star-Galaxy Separation

Star–galaxy separation becomes increasingly difficult at faint magnitudes ($i<24.0$ mag). In hscPipe, the star–galaxy classification primarily depends on the extendedness value, which is measured by the magnitude difference between PSF and cModel photometry in $i$-band. For more details about this algorithm, please see Bosch et al. (in prep.).

Fig 15 shows the current status of our star–galaxy classification scheme. Fig 15 shows the difference between PSF and cModel magnitudes as a function of input magnitude. We find that $20$% of synthetic stars at $i_{\mathrm{Input}}>23.0$ mag are misclassified as extended objects, while only a tiny fraction of faint synthetic galaxies ($i_{\mathrm{Input}}>24.0$) are misclassified as stars. This confirms that the current star–galaxy classification strategy in hscPipe results in a galaxy sample with high completeness and a star sample with high purity.

It is also clear that the seeing condition strongly modifies the distributions of $i_{\mathrm{PSF}}-i_{\mathrm{cModel}}$. Worse seeing condition makes star-galaxy separation more challenging. The completeness and purity of galaxy and star samples at the faint end will depend on seeing conditions.

In the next data release, hscPipe will include an improved star–galaxy classifier which will be based on a machine learning method that takes both size and color information into account.

6.3 Blending Effects

Fig 16 shows the $b$ parameter versus the difference between input and output values for both magnitudes and colors. For both PSF and cModel magnitude distributions, we can see clear trends that total fluxes are systematically underestimated for highly blended ($b>0.05$ for stars or $0.1$ for galaxies) objects. This effect is particularly strong for stars.

As for colors, higher values of $b$ results in higher statistical uncertainties for both stars and galaxies, but the trends are different for each. Highly blended stars have clearly redder $(g-i)$ colors than the input values. Highly blended galaxies seem to have $(g-i)$ colors that are bluer than the true values.

Blendedness-related uncertainties are not included in the photometric errors released by hscPipe. We therefore remind the HSC users to treat the highly blended object with greater caution.

6.4 Shape and Structural Parameters of Galaxies

In SDSS, structural parameters like size and shape from cModel are often used to study the evolution galaxies. Although the cModel in hscPipe follows a similar algorithm, its primary goal now is to provide accurate and consistent magnitude and color in all five bands for a vast majority of faint, small galaxies. To improve the stability of cModel for these low-$\mathrm{S}/\mathrm{N}$ and barely resolved galaxies, hscPipe uses Bayesian priors on galaxy sizes and axis ratios. However, the current implementation of these priors has a serious bug and leads to severe underestimated sizes and ellipticities of galaxies.

Fig 17 demonstrates this issue and compares the input and measured values of the cModel axis ratio. Here, $({\mathrm{b}/\mathrm{a}})_{\mathrm{CModel}}$ is the flux-weighted sum of the axis ratios estimated by exponential and de Vaucouleurs models.

Using our SynPipe tests, we discovered that the bug related to the priors only presents when usePixelWeight configuration is set to False during the data reduction. The parameter controls whether or not the per-pixel variance information is used during the model fitting process. As one can see, this leads to significantly overestimated axis ratios for almost all galaxies171717It also leads to underestimated half-light radius for the same galaxies.. We repeat the SynPipe run for galaxies with usePixelWeight$=$True, and this change clearly mitigates the problem. When we use usePixelWeight$=$True, $({\mathrm{b}/\mathrm{a}})_{\mathrm{CModel}}$ provides an unbiased axis ratio estimation for galaxies with $i_{\mathrm{Input}}<24.0$ mag. This change is still related to the bug in priors. The actual effect of using per-pixel variance should be very small. The next version of hscPipe will fix this bug, and SynPipe will help evaluate the precisions of shape and size measurements. This is a good example that demonstrates how the SynPipe framework is valuable for quality control and that can be used to improve hscPipe.

For the current HSC data release, we warn users against using the sizes and shapes from cModel to study the structural properties of galaxies.

7 Summary and Conclusion

In February 2017, the HSC survey made its first public data release, and the survey continues to produce increasingly larger amounts of high-quality imaging data. To help achieve the designed goals of the survey and to facilitate scientific investigations, we have developed SynPipe, a flexible framework based on hscPipe and that can be used to perform quality control on the HSC pipeline outputs. SynPipe operates by injecting synthetic stars and galaxies into single-visit images. These data are then processed by hscPipe to generate coadd images and to perform photometric measurements. With this approach, SynPipe can be used as an end-to-end and realistic test of the full data reduction process. In this paper, we use two tracts in the HSC Wide layer with representative seeing conditions (goodSeeing with FWHM$=0.45$′′ seeing; badSeeing with FWHM$=0.70$′′ seeing) to test the general behaviors of HSC photometry for both stars and galaxies.

Our main findings are the following:

The forced PSF photometry provides precise and accurate measurements of magnitude and color for isolated stars at $18.0<i<25.0$. The typical statistical uncertainties of HSC forced PSF photometry for stars ranges from 0.01 mag at $i{\sim}18.0$ mag to 0.08 mag at $i{\sim}25.0$ mag (1%-7% precision in the $i$-band). The forced PSF photometry can accurately recover the color–color sequences of stars. 2. 2.

The forced cModel photometry is reliable for synthetic galaxies with realistic distributions of structural parameters and colors at $20.0<i<25.0$ mag. The statistical uncertainties of forced cModel magnitude ranges from ${\sim}7$% at $i{\sim}20.0$ mag to ${\sim}10$% at $i{\sim}25.0$. The forced cModel colors for galaxies are precise and are consistent across in all five bands. They also do not show biases with input magnitudes and colors. 3. 3.

The forced PSF and cModel photometry are robust against moderate changes in the seeing conditions, despite the fact that worse seeing leads to a lower $\mathrm{S}/\mathrm{N}$ at fixed input magnitude. 4. 4.

Blending ($b>0.05$) has an impact on the accuracy of both PSF and cModel photometry, especially for stars. For highly blended stars, the forced PSF photometry overestimates the magnitudes of stars on average by 0.1–-0.2 mag. For galaxies, blending effects typically add an additional 0.05 mag statistical uncertainty in both magnitude and color estimates.

We also high-light several known issues with the current version of hscPipe:

The performance of the current star–galaxy separation algorithm in hscPipe is not perfect, and degrades at fainter magnitudes and with poor seeing conditions. The fraction of stars that are misclassified as extended objects is still quite high. $>20$% of the synthetic stars at $i>22.5$ mag are misclassified 181818Note that this does not mean high fraction of contamination in the galaxy sample. Please see Bosch et al. (in prep.) for more details. 2. 2.

The bug in priors on structural parameters and the usePixelWeights=False nominal setting in cModel result in in very biased estimates of the shapes and sizes for galaxies. The axis ratio from cModel is highly overestimated and the effective radius is typically underestimated. This issue hence can also affect the accuracy of star–galaxy classification.

The HSC collaboration is working on improving these issues for future versions of hscPipe and for subsequent data releases.

Here we focused on characterizing the photometric performance of hscPipe. However, SynPipe is a flexible tool and is being used for a range of other applications. For instance, Murata et al. (in prep.) uses SynPipe to characterize the level of blending in the HSC Wide Layer. (Niikura et al., 2017) apply SynPipe to high-cadence HSC observations of M31 HSC to search for microlensing events. SynPipe is also currently being used to estimate the completeness of high-$z$ LBG and Lyman-$\alpha$ emitters (Ono et al. in prep.; Konno et al. in prep.), to check the completeness of high-$z$ LBG pairs as a function of pair separation (Harikane et al. in prep.), to test the completeness of low surface–brightness dwarf galaxies selections (Greco et al. in prep.), and to test magnifications effects around nearby clusters (Chiu et al. in prep.).

Both hscPipe and SynPipe are undergoing active development. Our plan is to update SynPipe to provide a photometric benchmark for each HSC survey data release, which will help improve hscPipe. At the same time, we are also working on improving the efficiency of SynPipe. Many users are not necessarily concerned with the subtle effects involved in the image stacking process. Hence, a future implementation of SynPipe will include the option of injecting objects directly onto coadd images which will speed up the generation of synthetic data sets. The results shown in this paper demonstrate the utility of the SynPipe framework in order to perform end-to-end quality control on pipeline products to the level of accuracy needed by large panchromatic surveys.

{ack}

The Hyper Suprime-Cam (HSC) collaboration includes the astronomical communities of Japan and Taiwan, and Princeton University. The HSC instrumentation and software were developed by National Astronomical Observatory of Japan (NAOJ), Kavli Institute for the Physics and Mathematics of the Universe (Kavli IPMU), University of Tokyo, High Energy Accelerator Research Organization (KEK) in Japan, Academia Sinica Institute for Astronomy and Astrophysics (ASIAA) in Taiwan, and Princeton University in the United States. Funding was contributed by the FIRST program from Japanese Cabinet Office; Ministry of Education, Culture, Sports, Science and Technology (MEXT); Japan Society for the Promotion of Science (JSPS); Japan Science and Technology Agency (JST); Toray Science Foundation; NAOJ; Kavli IPMU; KEK; ASIAA; and Princeton University.

This paper makes use of software developed for the Large Synoptic Survey Telescope. We thank the LSST Project for making their code available as free software at http://dm.lsstcorp.org.

This work is in part supported by JSPS KAKENHI (Grant Number 26800093, 15H03654, and JP17H01131) as well as MEXT Grant-in-Aid for Scientific Research on Innovative Areas (No. 15H05887, 15H05892, 15H05893, 15K21733). RM is supported by the US Department of Energy Early Career Award Program. RyM is financially supported by the University of Tokyo-Princeton strategic partnership grant and Advanced Leading Graduate Course for Photon Science (ALPS).

Pan-STARRS1 Surveys (PS1) have been made possible through contributions of Institute for Astronomy, University of Hawaii, Pan-STARRS Project Office, Max-Planck Society and its participating institutes (Max Planck Institute for Astronomy, Heidelberg, and Max Planck Institute for Extraterrestrial Physics, Garching), Johns Hopkins University, Durham University, University of Edinburgh, Queen’s University Belfast, Harvard-Smithsonian Center for Astrophysics, Las Cumbres Observatory Global Telescope Network Incorporated, National Central University of Taiwan, Space Telescope Science Institute, National Aeronautics and Space Administration under Grant No. NNX08AR22G issued through Planetary Science Division of NASA Science Mission Directorate, National Science Foundation under Grant No. AST-1238877, University of Maryland, Eötvös Loránd University (ELTE), and Los Alamos National Laboratory.

This research made use of Astropy, a community-developed core Python package for Astronomy (Astropy Collaboration et al. 2013; http://www.astropy.org/); astroML, a machine learning library for astrophysics (VanderPlas et al. 2014; http://www.astroml.org/); SciPy, an open source scientific tool for Python (Jones et al. 2001; http://www.scipy.org/); NumPy, a fundamental package for scientific computing with Python (Walt et al. 2011; http://www.numpy.org/); Matplotlib, a 2-D plotting library for Python (Hunter 2007; http://matplotlib.org/); and scikit-learn, a machine learning library in Python (Pedregosa et al. 2011; http://scikit-learn.org/.

Appendix A Quality Control of Synthetic Objects

We apply quality control to the stars and galaxies selected from the HSC UltraDeep COSMOS fields before we match them with the stars and models for galaxies from the HST/ACS COSMOS data. The same quality control criteria are also used to select synthetic stars and galaxies from the SynPipe results before we perform photometric comparisons.

To select synthetic stars and galaxies from the HSC survey data in order to test the photometry, we apply some basic quality control cuts.

The HSC survey defines the full–depth & full–color regions (FDFC) to ensure the data used for science reach the expected number of exposures in each band ($\mathtt{(\#gri\ \geq\ 4)\ and\ (\#yz\ \geq\ 6)\ and\ (Limiting\ imag>25.6)}$; see Aihara et al. 2017a for details). Since the two tracts we used in this work are almost entirely covered in the FDFC region, we did not apply this cut.

Firstly, we make sure the object is a “primary” detection (in the inner part of tract and patch; not a parent in the deblending process), is successfully deblended, has reliable centroid, and is not bothered by various of optical issues:

• detect_is_primary==True
• deblend_skipped==False
• deblend_nchild=0
• [grizy]flags_badcentroid==False
• [grizy]centroid_sdss_flags==False
• [grizy]flags_pixel_edge==False
• [grizy]flags_pixel_interpolated_center==False
• [grizy]flags_pixel_saturated_center==False
• [grizy]flags_pixel_cr_center==False
• [grizy]flags_pixel_bad==False
• [grizy]flags_pixel_suspect_center==False
• [grizy]flags_pixel_clipped_any==False
• [grizy]flags_pixel_bright_object_center==False

For stars, we ensure the selected objects have useful PSF magnitude:

• [grizy]flux_psf_flags==False

For galaxies, we also make sure useful cModel photometry is available:

• [grizy]cmodel_flux_flags==False

Appendix B $b$: The Blendedness Parameter

To evaluate the degree to which an object is blended with others, hscPipe introduces the blendedness parameter: $b$ ([grizy]blendedness_abs_flux). We briefly define $b$ below; please refer to Bosch et al. (in prep.) and Murata et al. (in prep.) for more details.

For object ${\rm A}$:

[TABLE]

where $\mathcal{N}_{\rm{A}}({\bf x}\ |\ \mu_{\rm{A}},{\bf\Sigma}_{\rm{A}})$ is a 2-D Gaussian function at pixel position $\bf x$, $\mu_{\mathrm{A}}$ is the estimated centroid of object ${\rm A}$, and covariance ${\bf\Sigma}_{\mathrm{A}}$ is estimated based on the Gaussian-weighted adaptive second moments of object ${\rm A}$ (no PSF correction). $F_{\mathrm{total}}({\bf x})$ and $F_{\rm{A}}({\bf x})$ are pixel values of ${\rm A}$ before and after hscPipe deblending process, respectively.

By definition, the parameter is bound from 0 to 1. When the deblending process is carried out correctly, the $b({\rm A})$ parameter reflects the fraction of fluxes that comes from other objects in the region of ${\rm A}$.

The current version of hscPipe contains a bug related to the calculation of $b$ parameter. For bright object (e.g. $i<23.0$ mag), its effect can be ignored. For fainter object, it mostly leads to underestimate of $b$ parameter by $<12$%, especially for objects with $-3.5<\log_{10}b<-1.5$. This bug has been fixed for the future release of hscPipe and HSC survey data. For the SynPipe tests here, it does not qualitatively change the conclusions related to the highly-blended objects and about the relationship between $b$ and photometric precision. Please see Murata et al. (in prep.) and Bosch et al. (in prep.) for more details.