The FMOS-COSMOS survey of star-forming galaxies at $z\sim1.6$. V: Properties of dark matter halos containing H$\alpha$ emitting galaxies

TL;DR

This study analyzes the clustering and dark matter halo properties of star-forming galaxies at redshift 1.43-1.74 using FMOS-COSMOS data, revealing typical halo masses and the decline of stellar-to-halo mass ratio at lower halo masses.

Contribution

First to measure galaxy clustering and dark matter halo properties at this redshift using the FMOS-COSMOS survey with detailed halo occupation modeling.

Findings

Galaxies reside in halos with mass ~4.6×10^{12} h^{-1} M_⊙.

Clustering amplitude characterized by r_0 = 5.21 h^{-1} cMpc.

Stellar-to-halo mass ratio declines at M_h < 10^{12} M_⊙.

Abstract

We study the properties of dark matter halos that contain star-forming galaxies at $1.43 \le z \le 1.74$ using the FMOS-COSMOS survey. The sample consists of 516 objects with a detection of the H$\alpha$ emission line, that represent the star-forming population at this epoch having a stellar mass range of $10^{9.57}\le M_\ast/M_\odot \lesssim 10^{11.4}$ and a star formation rate range of $15\lesssim \mathrm{SFR}/(M_\odot \mathrm{yr^{-1}}) \lesssim 600$. We measure the projected two-point correlation function while carefully taking into account observational biases, and find a significant clustering amplitude at scales of $0.04$-$10~h^{-1}~\mathrm{cMpc}$, with a correlation length $r_0 = 5.21^{+0.70}_{-0.67}~h^{-1}~\mathrm{cMpc}$ and a bias $b=2.59^{+0.41}_{-0.34}$. We interpret our clustering measurement using a halo occupation distribution model. The sample galaxies appear to reside in halos with mass $M_\mathrm{h} = 4.6^{+1.1}_{-1.6}\times10^{12}~h^{-1}M_\odot$ on average that will likely become present-day halos of mass $M_\mathrm{h} (z=0) \sim2\times10^{13}~h^{-1}M_\odot$, equivalent to the typical halo mass scale of galaxy groups. We then confirm the decline of the stellar-to-halo mass ratio at $M_\mathrm{h}<10^{12}~M_\odot$, finding $M_\ast/M_\mathrm{h} \approx 5\times10^{-3}$ at $M_\mathrm{h}=10^{11.86}~M_\odot$, which is lower by a factor of 2-4 than those measured at higher masses. Finally, we use our results to illustrate the future capabilities of Subaru's Prime-Focus Spectrograph, a next-generation instrument that will provide strong constraints on the galaxy-formation scenario by obtaining precise measurements of galaxy clustering at $z>1$.

Tables

Table 1. Table 1 Symbols used in this paper

Symbol	Definition
$M_h$	halo mass
$M_{\ast }$	stellar mass
$M_{\ast }^{\lim }$	stellar mass limit for our sample $\left({10}^{9.57}{M}_{\odot }\right)$
$\mathrm{SFR}$	star formation rate
$f_{H\alpha }^{\mathrm{pre}}$	predicted H$\alpha$ flux (after dust obscuration)
$f_{H\alpha }^{\lim }$	lower limit of $f_{H\alpha }^{\mathrm{pre}}$ for our sample $\left(1\times {10}^{-16}\mathrm{erg}{\mathrm{cm}}^{-2}{s}^{-1}\right)$
$f_{\mathrm{fake}}$	fraction of misidentified objects in our sample
$\xi$, ${\xi }_{\mathrm{dm}}$	real space correlation function of galaxies/dark matter
$w_p$, $w_{p,\mathrm{dm}}$	projected correlation function of galaxies/dark matter
$r_p$	comoving tangential separation
$\pi$	comoving line-of-sight separation
$b$	galaxy bias
$\omega \left(\theta \right)$	angular correlation function

Table 2. Table 2 Galaxy samples

Sample name	$N$aaNumber of galaxies in each sample.	$n\left({10}^{-3}{h}^3{\mathrm{cMpc}}^{-3}\right)$	Selection criteria	Mean $M_{\ast }$bbMean and median stellar masses of each sample, expressed in $\log M_{\ast }/M_{\odot }$.	Median $M_{\ast }$bbMean and median stellar masses of each sample, expressed in $\log M_{\ast }/M_{\odot }$.
$M_{\ast }$-selected	7006	9.29ccNumber density of photometrically-selected sample, computed in the volume over $1.46\le z\le 1.72$ ($7.54\times {10}^5{h}^{-3}{\mathrm{cMpc}}^3$).	$1.46\le z_{\mathrm{phot}}\le 1.72$, $M_{\ast }\ge {10}^{9.57}{M}_{\odot }$	10.42	10.04
$M_{\ast }+K_S$-selected	6453	8.55ccNumber density of photometrically-selected sample, computed in the volume over $1.46\le z\le 1.72$ ($7.54\times {10}^5{h}^{-3}{\mathrm{cMpc}}^3$).	$+$ $K_S\le 23.5$	10.45	10.10
FMOS-parent	2319	3.07ccNumber density of photometrically-selected sample, computed in the volume over $1.46\le z\le 1.72$ ($7.54\times {10}^5{h}^{-3}{\mathrm{cMpc}}^3$).	$+$ Predicted $f\left(H\alpha \right)\ge {10}^{-16}\mathrm{erg}{s}^{-1}{\mathrm{cm}}^{-2}$	10.58	10.20
FMOS-fiber-target	1182	–	$+$ Observed	10.59	10.23
FMOS-spec-$z$	516	0.575ddNumber density of spectroscopic sample, computed in the volume over $1.43\le z\le 1.74$ ($8.98\times {10}^5{h}^{-3}{\mathrm{cMpc}}^3$).	$+$ H$\alpha$-detected at $1.43\le z\le 1.74$	10.54	10.22

Table 3. Table 3 Parameter constraints

Model	Params.	PrioraaPrior probability distribution for each parameter. $G(x_0,\sigma )$ denotes a Gaussian function with a mean $x_0$ and standard deviation $\sigma$.	Best-fitbbThe best-fitting model parameters that gives the minimum ${\chi }^2$ with the broad prior on $f_{\mathrm{fake}}$ (see Section 4.3).	Best-fit ($f_{\mathrm{fake}}=0$)ccThe best-fitting parameters for $f_{\mathrm{fake}}=0$.
Power-law	$r_0/\left(h^{-1}\mathrm{cMpc}\right)$	–	${5.21}_{-0.67}^{+0.70}$	${4.46}_{-0.56}^{+0.56}$
	$\gamma$	–	${1.99}_{-0.17}^{+0.13}$	${1.99}_{-0.21}^{+0.15}$
	$f_{\mathrm{fake}}$	$\ge 0,G(0.14,0.06)$	${0.14}_{-0.06}^{+0.06}$	–
Dark matter	$b$	–	${2.59}_{-0.34}^{+0.41}$	${2.23}_{-0.34}^{+0.36}$
	$f_{\mathrm{fake}}$	$\ge 0,G(0.14,0.06)$	${0.14}_{-0.06}^{+0.06}$	–

Table 4. Table 4 Priors and constraints of the HOD parameters

Parameters	PrioraaPrior probability distribution of each parameter. $U(x_1,x_2)$ is a non-informative prior with a interval [$x_1$, $x_2$]. $G(x_0,\sigma )$ is a Gaussian function with a mean $x_0$ and standard deviation $\sigma$.	Best-fit	Best-fit ($f_{\mathrm{fake}}$=0)
${\chi }^2/\nu$	–	1.23	1.01
$\log M_{\min }/\left(h^{-1}{M}_{\odot }\right)$	$U(9,15)$	${11.71}_{-0.12}^{+0.11}$	${11.65}_{-0.10}^{+0.09}$
${\sigma }_{\log M}^2$	$\ge 0,G(0.24,0.03)$ for ${\sigma }_{\log M}$	${0.057}_{-0.021}^{+0.020}$	${0.057}_{-0.024}^{+0.022}$
$\log M_1^{\prime }/\left(h^{-1}{M}_{\odot }\right)$	$U(9,15)$	${12.28}_{-0.20}^{+0.18}$	${12.40}_{-0.23}^{+0.22}$
$f_{\mathrm{fake}}$	$\ge 0,G(0.14,0.06)$	${0.099}_{-0.092}^{+0.053}$	—
Inferred quantities	Prior	Best-fit	Best-fit ($f_{\mathrm{fake}}=0$)
$n_{\mathrm{tot}}/{\left(h^{-1}\mathrm{cMpc}\right)}^3$bbObserved number density of the $M_{\ast }$-selected sample. This is independently constrained from the photometric catalog.	$>0,G\left(\mathrm{9.29\hspace{0.17em}0.93}\right)$	${9.27}_{-1.40}^{+1.21}\times {10}^{-3}$	${9.22}_{-1.56}^{+1.49}\times {10}^{-3}$
$n_{\mathrm{cen}}/{\left(h^{-1}\mathrm{cMpc}\right)}^3$	–	${5.74}_{-1.67}^{+1.59}\times {10}^{-3}$	${6.70}_{-1.68}^{+1.61}\times {10}^{-3}$
$n_{\mathrm{sat}}/{\left(h^{-1}\mathrm{cMpc}\right)}^3$	–	${3.53}_{-2.10}^{+1.30}\times {10}^{-3}$	${2.52}_{-1.94}^{+0.94}\times {10}^{-3}$
$f_{\mathrm{sat}}$	–	${0.38}_{-0.20}^{+0.14}$	${0.27}_{-0.19}^{+0.10}$
$M_{\mathrm{eff}}/\left(h^{-1}{M}_{\odot }\right)$	–	${4.62}_{-1.63}^{+1.14}\times {10}^{12}$	${3.78}_{-1.45}^{+0.95}\times {10}^{12}$
$b_{\mathrm{eff}}$	–	${2.17}_{-0.24}^{+0.17}$	${2.04}_{-0.22}^{+0.14}$
$\log M_1/M_{\min }$	–	${0.66}_{-0.27}^{+0.30}$	${0.83}_{-0.26}^{+0.31}$

Table 5. Table 5 Mock samples from the ν 2 superscript 𝜈 2 \nu^{2} GC simulation

Sample name	$N$	ConditionsaaThe thresholds on $V_{\max }$ and the realistic shape of the FMOS survey area are applied for all the mock samples listed here.
Mock-parent	$\sim 2300$	$1.46\le z_{\mathrm{phot}}\le 1.72$bbPhotometric redshift of a (sub)halo is generated by addition a random error on the true redshift.
Mock-zweight	516	$1.46\le z_{\mathrm{phot}}\le 1.72$, $1.43\le z_{\mathrm{spec}}\le 1.74$, and non-uniform detectabilityddThe inhomogeneous weight function shown in Figure 7 is applied.
Mock-fiber-target	$\sim 1200$	$1.46\le z_{\mathrm{phot}}\le 1.72$ and fiber allocation
Mock-spec$z$	516	$1.46\le z_{\mathrm{phot}}\le 1.72$, fiber allocation, and $1.43\le z_{\mathrm{spec}}\le 1.74$

Equations

$$f_{\mathrm{H}\alpha }^{\mathrm{pre}}=\frac{1}{4\pi d_{\mathrm{L}}^2}\frac{\mathrm{SFR}(M_{\odot }\mathrm{y}{\mathrm{r}}^{-1})}{4.65\times 10^{-42}}10^{-0.4A_{\mathrm{H}\alpha }},$$

$$\xi \left(r\right)=\frac{N_R(N_R-1)}{N_D(N_D-1)}\frac{DD\left(r\right)}{RR\left(r\right)}-2\frac{N_R-1}{N_D}\frac{DR\left(r\right)}{RR\left(r\right)}+1,$$

$$\pi \equiv |\vec{s}\cdot \vec{l}|/|\vec{l}|,r_{\mathrm{p}}\equiv \sqrt{|\vec{s}{|}^2-{\pi }^2}.$$

$$w_{\mathrm{p}}\left(r_{\mathrm{p}}\right)=2{\int }_0^{{\pi }_{\max }}\xi \left(r_{\mathrm{p}},\pi \right)d\pi .$$

$${\mathbb{C}}_{ij}=\frac{N-1}{N}\sum _{k=1}^N\left[X_i^k-⟨X_i⟩\right]\left[X_j^k-⟨X_j⟩\right],$$

$${\left({\sigma }_i^2\right)}^{\mathrm{smooth}}=({\mathbb{C}}_{i-1,i-1}+2{\mathbb{C}}_{ii}+{\mathbb{C}}_{i+1,i+1})/4.$$

$$\frac{1}{16}\left[\begin{array}[]{ccc}1&2&1\\ 2&4&2\\ 1&2&1\\ \end{array}\right].$$

$$\mathbb{C}^{\mathrm{smooth}}_{ij}=\left\{\begin{array}[]{ll}\left(\sigma_{i}^{2}\right)^{\mathrm{smooth}}&(i=j),\\ \mathbb{R}^{\mathrm{smooth}}_{ij}\sqrt{(\sigma^{2}_{i})^{\mathrm{smooth}}(\sigma^{2}_{j})^{\mathrm{smooth}}}&(i\neq j).\\ \end{array}\right.$$

$$\mathcal{P}\propto \exp \left(-\frac{{\chi }_{\mathrm{data}}^2+{\chi }_{\mathrm{prior}}^2}{2}\right).$$

$${\chi }_{\mathrm{data}}^2=\sum _{i=0}^N\sum _{j=0}^N\left[X_i^{\mathrm{mod}}-X_i^{\mathrm{obs}}\right]({\mathbb{C}}_{ij}^{\mathrm{smooth}}{)}^{-1}\left[X_j^{\mathrm{mod}}-X_j^{\mathrm{obs}}\right]$$

$$X_i^{\mathrm{mod}}=\log \left[{\int }_{r_{\min }}^{r_{\max }}{w}_{\mathrm{p}}^{\mathrm{mod}}(r_{\mathrm{p}})d{r}_{\mathrm{p}}/(r_{\max }-r_{\min })\right].$$

$$f\left(\theta \right)=\frac{1+{\omega }_{\mathrm{par}}\left(\theta \right)}{1+{\omega }_{\mathrm{tar}}\left(\theta \right)}.$$

$$w_{\mathrm{p}}\left(r_{\mathrm{p}}\right)=w_{\mathrm{p}}^{\mathrm{obs}}\left(r_{\mathrm{p}}\right)+C.$$

$$C_{\xi }=\frac{{\sum }_i{\xi }^{\mathrm{mod}}\left(r_i\right)RR\left(r_i\right)}{{\sum }_{i}RR\left(r_i\right)},$$

$$C=2{\int }_0^{{\pi }_{\max }}{C}_{\xi }d\pi =2C_{\xi }\times 30h^{-1}\mathrm{cMpc}.$$

$$w_{\mathrm{p}}\left(r_{\mathrm{p}}\right)=r_{\mathrm{p}}{\left(\frac{r_{\mathrm{p}}}{r_0}\right)}^{-\gamma }\frac{\Gamma \left(\frac{1}{2}\right)\Gamma \left(\frac{\gamma -1}{2}\right)}{\Gamma \left(\frac{\gamma }{2}\right)},$$

$$\xi (r)=b^2{\xi }_{\mathrm{dm}}(r),$$

$$w_{\mathrm{p}}(r_{\mathrm{p}})=b^2{w}_{\mathrm{p},\mathrm{dm}}(r_{\mathrm{p}}),$$

$$⟨N|M,z⟩=⟨N_{\mathrm{cen}}|M,z⟩+⟨N_{\mathrm{sat}}|M,z⟩.$$

$$n_{\mathrm{tot}}=\int ⟨N|M,z⟩n(M,z)\mathrm{d}M,$$

$$\xi (r)={\xi }^{1h}(r)+{\xi }^{2h}(r),$$

$$\xi (r)=\frac{1}{2{\pi }^2}{\int }_0^{\infty }\mathrm{d}k{k}^{2}P(k)\frac{\sin kr}{kr},$$

$$P(k)=P^{1h}(k)+P^{2h}(k).$$

$$P^{1h}(k)=\frac{1}{n_{\mathrm{tot}}^2}\int n\left(M\right)\mathrm{d}M$$

$$\left[2⟨N_{\mathrm{cen}}|M⟩⟨N_{\mathrm{sat}}|M⟩u(k,M)+{⟨N_{\mathrm{sat}}|M⟩}^2{u}^2(k,M)\right],$$

$$P^{2h}(k)=\frac{1}{n_{\mathrm{tot}}^2}\int \mathrm{d}{M}_1\int \mathrm{d}{M}_{2}n\left(M_1\right)n\left(M_2\right)$$

$$[⟨N_{\mathrm{cen}}|M_2⟩⟨N_{\mathrm{cen}}|M_2⟩+2⟨N_{\mathrm{cen}}|M_1⟩⟨N_{\mathrm{sat}}|M_2⟩u(k,M_2)$$

$$+⟨N_{\mathrm{sat}}|M_1⟩⟨N_{\mathrm{sat}}|M_2⟩u(k,M_1)u(k,M_2)]P_{\mathrm{hh}}(k|M_1,M_2),$$

$$⟨N_{\mathrm{cen}}|M⟩=\frac{1}{2}\left[1+\mathrm{erf}\left(\frac{\log M-\log M_{\min }}{{\sigma }_{\log M}}\right)\right],$$

$$⟨N_{\mathrm{sat}}|M⟩=\frac{1}{2}\left[1+\mathrm{erf}\left(\frac{\log (M/M_{\min })}{{\sigma }_{\log M}}\right)\right]{\left(\frac{M-M_{\mathrm{cut}}}{M_1^{\prime }}\right)}^{\alpha }.$$

$$b_{\mathrm{eff}}=\frac{1}{n_{\mathrm{tot}}}\int b_h(M)⟨N|M⟩n\left(M\right)\mathrm{d}M$$

$$f_{\mathrm{sat}}=\frac{1}{n_{\mathrm{tot}}}\int ⟨N_{\mathrm{sat}}|M⟩n\left(M\right)\mathrm{d}M$$

Full text

1. Introduction

According to our current cosmological model (e.g., White & Rees, 1978; Springel et al., 2006; Frieman et al., 2008), the formation of structure in the Universe is dictated by cold dark matter and dark energy. On large scales, matter is organized in cosmic web-like structures consisting of dense nodes, filaments, sheets and voids. Dark matter virializes into extended halos at the density peaks within this structure. Galaxies form within these halos as a result of complex baryonic processes such as star formation and feedback (Rees & Ostriker, 1977; White & Rees, 1978). Understanding the connection between galaxies and dark matter halos can thus shed light on the physical processes that lead to the formation and evolution of galaxies. Specifically, the mass of halos sets their global properties such as the abundance and spatial distribution (Press & Schechter, 1974; Cole & Kaiser, 1989; Mo & White, 2002) as well as the evolution of their constituent galaxies. A number of different methods have been used to infer halo masses and constrain the connection between galaxies and dark matter halo masses. Direct techniques such as measurements of galaxy rotation curves (e.g., Rubin et al., 1980, 1982), X-ray emission from the hot intracluster gas (e.g., Rykoff et al., 2008) and statistical techniques such as the kinematics of satellite galaxies (e.g., More et al., 2009, 2011) and weak gravitational lensing (e.g., Hoekstra et al., 2005; Mandelbaum et al., 2006; Cacciato et al., 2009; Leauthaud et al., 2011; Han et al., 2015; Mandelbaum et al., 2016; van Uitert et al., 2016) can be used to evaluate dynamical mass of systems, albeit all restricted to fairly local redshifts. At redshifts greater than unity, one has to resort to indirect techniques such as subhalo abundance matching to infer the galaxy–dark matter connection (e.g., Kravtsov et al., 2004; Conroy et al., 2006; Kravtsov et al., 2006; Moster et al., 2010; Behroozi et al., 2010; Masaki et al., 2013).

Analyses of spatial distribution of galaxies can also help to constrain the galaxy–dark matter connection as galaxies reflect the spatial clustering properties of the halos. For a given cosmological model, the clustering of dark matter halos depends upon halo mass, such that more massive halos display stronger clustering than less massive halos. Therefore, the halo masses can be inferred from the observed clustering amplitude of the hosted galaxies. Large galaxy surveys such as the two-degree-Field Galaxy Redshift Survey (Colless et al., 2001) and Sloan Digital Sky Survey (SDSS; York et al., 2000) have successfully enabled studies of galaxy clustering at low redshift ($z\sim 0.1$; e.g., Park et al. 1994; Guzzo et al. 2000; Norberg et al. 2001; Zehavi et al. 2002). These studies have also shown that galaxy clustering depends on galactic properties, such as luminosity, color, and morphology, such that relatively luminous (i.e., massive), redder, and bulge-dominated galaxies cluster more strongly (or live in more massive halos), while less massive, bluer, or disk-dominated galaxies have a weaker clustering signal (e.g., Norberg et al., 2002; Zehavi et al., 2005, 2011). Beyond the local Universe ($z\gtrsim 0.5$), most studies of galaxy clustering have measured angular correlation functions based on photometric redshift or color selection (e.g., Ouchi et al., 2004; Adelberger et al., 2005; Hartley et al., 2008; McCracken et al., 2010; Wake et al., 2011; Coupon et al., 2012; McCracken et al., 2015), while measurements based on spectroscopic samples have been limited (e.g., Meneux et al., 2008, 2009; Abbas et al., 2010; Coupon et al., 2015; Durkalec et al., 2015a).

The halo occupation distribution (HOD) framework, which describes the average number of galaxies hosted by a halo as a function of halo mass, has been routinely used to describe the observed abundance and clustering of galaxies (e.g., Peacock & Smith, 2000; Scoccimarro et al., 2001; Berlind & Weinberg, 2002; Berlind et al., 2003; Kravtsov et al., 2004; Zheng et al., 2005). These observables constrain key quantities of the HOD, such as the minimum mass of halos that host at least one galaxy for a specific population. The HOD models have been applied to interpret galaxy clustering in the local Universe (e.g., Zehavi et al., 2004; Zheng et al., 2005, 2007; Cacciato et al., 2009, 2013; Zehavi et al., 2011; Leauthaud et al., 2011; More et al., 2015) and at higher redshifts up to $z\sim 7$ (e.g., Wake et al., 2011; Geach et al., 2012; Durkalec et al., 2015a, b; Harikane et al., 2016). However, most analyses of the HOD of high-redshift ($z\gtrsim 1$) galaxies have been conducted by using photometric samples (e.g., Wake et al., 2011; Coupon et al., 2012; Martinez-Manso et al., 2015; McCracken et al., 2015). While imaging surveys provide large and deep samples at a much lower cost compared to spectroscopy, it is hard to resolve the redshift evolution due to the contamination of back/foreground objects and the associated large uncertainties in the redshift determination. For example, even for the estimates in the COSMOS catalogs based on 30-band photometry, the typical error is about $\Delta z/(1+z)\sim 0.03$ (Ilbert et al., 2013; Laigle et al., 2016).

On the other hand, spectroscopic surveys at higher redshifts generally tend to be restricted to relatively bright, rare objects. Such samples may not represent the general galaxy population. Therefore, spectroscopic studies based on galaxies representative of the epoch’s average population are required. It has been established that the bulk of star-forming galaxies follow a tight correlation between stellar mass ($M_{\ast}$) and star formation rate (SFR) over a wide range of redshift up to $z\sim 4$ (e.g., Noeske et al., 2007; Daddi et al., 2007; Whitaker et al., 2012; Kashino et al., 2013; Daddi et al., 2009; Pannella et al., 2015) or even higher (e.g., Salmon et al., 2015), the so-called “main sequence”. Galaxies along this sequence dominate the cosmic stellar mass density at $z\gtrsim 1$ (Ilbert et al., 2013) as well as the cosmic star formation rate density over cosmic history (Rodighiero et al., 2011; Schreiber et al., 2015). Therefore, main-sequence galaxies are representative of the average star-forming galaxy population at all redshifts.

In this work, we investigate the properties of halos that host galaxies representative of the main-sequence star-forming population using spectroscopic selection of 516 objects at $1.43\leq z\leq 1.74$ down to a stellar mass $10^{9.57}~{}M_{\odot}$. This redshift range marks the transition epoch of cosmic star formation from its peak to the decline phase that continues until today (Madau & Dickinson, 2014). Galaxies in our sample are drawn from a near-IR spectroscopic campaign using the Fiber Multi-Object Spectrograph (FMOS) on the Subaru telescope, called the FMOS-COSMOS survey (Silverman et al., 2015). For this sample, we successfully map the small scale structures below $1~{}h^{-1}~{}\mathrm{comoving~{}Mpc~{}(cMpc)}$, where the contribution from galaxies that reside in the same halo (i.e., one-halo term) becomes important. Such a high sampling rate per unit area is one of the unique advantages of our survey and makes it complementary to the FastSound survey, which is another wide-field spectroscopic survey carried out with FMOS (Tonegawa et al., 2015; Okumura et al., 2016).

The paper is organized as follows. In Section 2, we provide an overview of the FMOS-COSMOS survey and describe our galaxy sample. We describe the methods employed to measure clustering in Section 3, and corrections for critical biases in Section 4. We present our clustering measurements in Section 5. Results are interpreted using an HOD modeling in Sections 6, and we discuss the physical implications of the derived quantities in Section 7. We finally summarize our results and conclusions in Section 8. Throughout the paper, magnitudes are given in the AB system and a flat $\Lambda$CDM cosmology with $\left(\Omega_{m},\Omega_{\Lambda}\right)=\left(0.3,0.7\right)$ is assumed. We express distances in comoving units and halo masses with the Hubble parameter $h$ in units of $100~{}\mathrm{km~{}s^{-1}~{}Mpc^{-1}}$, while stellar masses and, subsequently, the stellar mass-to-halo mass ratios are computed assuming $h=0.7$. We use a Chabrier (2003) initial mass function (IMF). The conversion factor to a Chabrier IMF is $1/1.7$ for both stellar masses and SFRs given in our companion papers that use a Salpeter IMF (Kashino et al., 2013, 2017; Silverman et al., 2015). We use “$\log$” to denote a logarithm with a base 10 ($\log_{10}$). For references, Table 1 defines symbols used in this paper.

2. Data

2.1. Overview of the FMOS-COSMOS survey

The galaxy sample used in this paper is constructed from the data set of the FMOS-COSMOS survey. Details of the observations, survey design and data analysis are described elsewhere (Kashino et al., 2013; Silverman et al., 2015). FMOS is a near-infrared spectrograph on the Subaru telescope with high multiplex capabilities as it allows the placement of 400 fibers, each with an aperture of $1.2\arcsec$ in diameter, over a circular field of 0.19 square degrees (Kimura et al., 2010). In cross-beam switching mode, roughly 200 objects can be observed at a given time. FMOS has an OH airglow suppression system that blocks the strong OH emission lines ($\sim 30\%$ of the entire spectral window).

The survey is designed to detect the H$\alpha$ emission line with the $H$-long grating (1.6–1.8$~{}\mathrm{\mu m}$, spectral resolution $R\approx 3000$). The H$\alpha$ and [Nii]$\lambda\lambda$6548, 6584 lines are well separated and the accuracy of redshift determination is $\Delta z/(1+z)=2.2\mathrm{e-4}$, corresponding to $30~{}\mathrm{km~{}s^{-1}}$ at $z\sim 1.6$ (Silverman et al., 2015). We carried out additional observations with the $J$-long grating (1.11–1.35 $\mu$m) to detect the H$\beta$ and [Oiii]$\lambda\lambda$5007, 4959 emission lines to establish the ionization conditions of $z\sim 1.6$ star-forming galaxies (Zahid et al., 2014; Kartaltepe et al., 2015; Kashino et al., 2017) and confirm redshift measurements for about half of the sample. Our survey covers the entire COSMOS field with multiple FMOS footprints having some overlap regions. All data are reduced with the FMOS Image-Based Reduction Package (FIBRE-pac; Iwamuro et al. 2012).

2.2. Sample selection

In this study, we use a catalog of galaxies with spectroscopic redshifts based on observations carried out from 2012 March to 2014 February, including 11 and 6 pointings in the central region of the COSMOS field with the $H$- and $J$-long gratings, respectively. The survey field is shown in Figure 1 and has a area of $0.810~{}\mathrm{deg}^{2}$. We remove the sky regions impacted by bright stars (shaded areas), which account for 4.9% of the survey area.

Our sample is selected from the COSMOS photometric catalog that includes the Ultra-VISTA/VIRCAM photometry (McCracken et al., 2012; Ilbert et al., 2013). Photometric redshifts and stellar masses are derived by spectral energy distribution (SED) fitting with the Le Phare photometric code (Arnouts & Ilbert, 2011), assuming Bruzual & Charlot (2003) stellar population synthesis models and a Chabrier IMF. The photometric catalog reaches a magnitude limit of $K_{\mathrm{S}}\approx 24~{}\mathrm{mag}$, thus being complete for more than $90\%$ of the galaxies down to a stellar mass $10^{9.57}M_{\odot}$ (Ilbert et al., 2013). We also derive star formation rates (SFRs) from SED fitting assuming a constant star formation history, as extensively described in Silverman et al. (2015), and compute the predicted H$\alpha$ flux ($f^{\mathrm{pre}}_{\mathrm{H\alpha}}$) for each galaxy using a relation from Kennicutt (1998), converted for use with a Chabrier IMF:

[TABLE]

where $d_{\mathrm{L}}$ is luminosity distance. The values of $f^{\mathrm{pre}}_{\mathrm{H\alpha}}$ represent the total light coming from galaxies, as opposed to only that seen by a single fiber. Dust extinction of the stellar component $E_{\mathrm{star}}(B-V)$ is also derived from our SED fitting. This value has been converted to a nebular extinction to the H$\alpha$ emission as $A_{\mathrm{H\alpha}}=3.325E_{\mathrm{star}}(B-V)/0.66$ (see e.g., Kashino et al. 2013), assuming a Calzetti et al. (2000) extinction curve. For the remainder of the paper, these SED-based quantities are used for selection, while the spectroscopic redshifts and observed H$\alpha$ fluxes are incorporated into the final determination of their SFRs (Section 7.3). We describe in detail the steps of our sample selection below. Each subsample will be labeled at various stages of the selection to make the treatment of biases clear.

Over an effective survey area ($0.77~{}\mathrm{deg}^{2}$), we find 7006 galaxies in the COSMOS catalog that have stellar mass above a threshold mass $M_{\ast}^{\mathrm{lim}}\equiv 10^{9.57}~{}M_{\odot}$ and photometric redshift ($z_{\mathrm{phot}}$) between 1.46 and 1.72. We refer to these mass (and photo-$z$) selected galaxies as $M_{\ast}$-selected sample. Our targets are restricted to those with the Ultra-VISTA/VIRCAM photometry $K_{\mathrm{S}}\leq 23.5$. We designate the subset of 6453 galaxies with both $M_{\ast}\geq M_{\ast}^{\mathrm{lim}}$ and $K_{\mathrm{S}}\leq 23.5$ as $M_{\ast}+K_{\mathrm{S}}$-selected sample. In addition, we impose a limit on the predicted H$\alpha$ flux computed from the SED-based SFR and dust extinction (Equation 1), as $f^{\mathrm{pre}}_{\mathrm{H\alpha}}\geq f^{\mathrm{lim}}_{\mathrm{H\alpha}}\equiv 1\times 10^{-16}~{}\mathrm{erg~{}cm^{-2}~{}s^{-1}}$, to achieve an acceptable success rate of detecting the H$\alpha$ emission line with FMOS. This flux limit is equivalent to $\mathrm{SFR}\approx 20~{}M_{\odot}~{}\mathrm{yr^{-1}}$ for galaxies with typical extinction $A_{\mathrm{H\alpha}}=1$ mag at $z=1.6$. Of the $M_{\ast}+K_{\mathrm{S}}$-selected sample, we find 2139 galaxies having $f^{\mathrm{pre}}_{\mathrm{H\alpha}}\geq f^{\mathrm{lim}}_{\mathrm{H\alpha}}$, which are referred to as FMOS-parent sample. This sample is used as the input catalog for the process of allocating fibers to galaxies.

From the FMOS-parent sample, we observed 1182 galaxies with the $H$-long grating, which are referred to as FMOS-fiber-target sample. Of these, we select 516 galaxies that have a positive H$\alpha$ detection ($S/N\geq 1.5$) in the $H$-long window ($1.43\leq z\leq 1.74$; median $z=1.588$) to measure their spatial clustering. This final sample, refereed to as FMOS-spec-$z$ sample, is restricted to galaxies having one or more emission lines with $S/N\geq 3$. In most cases (503/516), H$\alpha$ is detected at $S/N\geq 3$. For H$\alpha$ detections of low significance ($1.5\leq S/N<3$), we require the detection of at least one other line (e.g., [O iii]$\lambda$5007) at $S/N\geq 3$. Table 2 summarizes the size, number density, selection criteria and mean/median stellar masses of each subsample. The total survey volume over the range of spectroscopic redshift ($1.43\leq z\leq 1.74$) is $8.98\times 10^{5}~{}(h^{-1}~{}\mathrm{cMpc})^{3}$.

2.3. Sample characteristics

In Figure 2, we show the SFR derived from SED as a function of $M_{\ast}$ for galaxies in our sample. We note that stellar masses and SFRs of the FMOS-spec-$z$ sample shown here are not revised by incorporating their spectroscopic redshifts, but rather the original values based on photometric redshifts used for the sample selection. In agreement with many studies, there is a clear correlation between SFR and $M_{\ast}$, i.e., the star-forming main sequence. It is evident that the our sample traces the underlying distribution of star-forming galaxies. We find no significant difference in SFR between the FMOS-parent and FMOS-spec-$z$ samples. However, it is shown that galaxies having relatively low SFRs (64%) are missed in the selection due to the self-imposed limitation on the predicted H$\alpha$ flux. The median SFRs and the 68th percentiles are shown in eight bins of stellar mass for both the $M_{\ast}+K_{\mathrm{S}}$-selected (triangles) and FMOS-spec-$z$ (blue squares) samples. The FMOS-spec-$z$ sample is on average biased toward higher SFRs by $0.1\textrm{--}0.2~{}\mathrm{dex}$ at $M_{\ast}\lesssim 10^{10.8}~{}M_{\odot}$. However, such a bias is only about half the scatter in SFR of the $M_{\ast}+K_{\mathrm{S}}$-selected sample. Therefore, we contend that the FMOS-spec-$z$ sample is well representative of the normal star-forming population at these redshifts.

We assess the completeness of our sample at a given stellar mass in Figure 3, which compares the stellar mass distributions of the $M_{\ast}$-selected, $M_{\ast}+K_{\mathrm{S}}$-selected and FMOS-parent samples. In the upper panel, the fractions of the number of galaxies in the $M_{\ast}+K_{\mathrm{S}}$-selected or in the FMOS-parent samples to the number of galaxies in the $M_{\ast}$-selected sample are shown as a function of stellar mass. As evident, approximately 8% of galaxies in the $M_{\ast}$-selected sample are missed near the lower mass limit ($M_{\ast}\lesssim 10^{10}~{}M_{\odot}$) by the inclusion of the $K_{\mathrm{S}}\leq 23.5$ limit, while the stellar mass completeness reaches almost unity at $M_{\ast}>10^{10.1}~{}M_{\odot}$. In the FMOS-parent sample, a large fraction of galaxies in the $M_{\ast}$-selected sample are missed due to the limit on the predicted H$\alpha$ flux. The sampling rate increases slowly with increasing stellar mass. As a consequence, the FMOS-parent sample has slightly higher mean and median stellar masses as compared to the $M_{\ast}$-selected or $M_{\ast}+K_{\mathrm{S}}$-selected sample (Table 2). Given an expected dependence of the clustering amplitude on stellar mass, such a selection bias may affect the observed correlation function. The effects of stellar mass incompleteness of our sample are further discussed using mock samples in Appendix B. We ensure that such effects do not significantly impact our results and conclusions.

We further evaluate the observational selection effects. Figure 4 shows the distribution of photometric redshift, stellar mass, SED-based SFR, and the predicted H$\alpha$ flux for the FMOS-parent, -fiber-target, and -spec-$z$ samples. These values are not revised for spectroscopic redshifts. The fractions of the number of galaxies in the FMOS-fiber-target sample or in the FMOS-spec-$z$ sample to the number of galaxies in the FMOS-parent sample are presented in the upper panels. We highlight that the sampling rate is almost uniform as a function of photometric redshift, stellar mass and SFR. As a consequence, the median values of these quantities of the FMOS-spec-$z$ sample are in good agreement with those of the FMOS-parent sample (see Table 2 for average masses). In contrast, the right panel of Figure 4 indicates that the FMOS-spec-$z$ sample is slightly biased towards having higher $f^{\mathrm{pre}}_{\mathrm{H\alpha}}$ as compared to the FMOS-parent sample. Such a bias is expected from the fact that stronger H$\alpha$ lines are easier to detect. Even though, the median value of the predicted H$\alpha$ flux of the FMOS-spec-$z$ sample ($\log f^{\mathrm{pre}}_{\mathrm{H\alpha}}/(\mathrm{erg~{}s^{-1}~{}cm^{-2}})=-15.79$) is close to that of the FMOS-parent sample ($\log f^{\mathrm{pre}}_{\mathrm{H\alpha}}/(\mathrm{erg~{}s^{-1}~{}cm^{-2}})=-15.83$). Therefore, we conclude that the observational sampling biases do not significantly affect the characteristics of the spec-$z$ sample, relative to the parent sample.

In Figure 5, we show the distribution of spectroscopic redshifts for 516 galaxies in the FMOS-spec-$z$ sample with positions of OH lines highlighted. Wavelengths of OH lines are converted into redshifts based on the wavelength of the H$\alpha$ emission line as $z=\lambda_{\mathrm{OH}}/\lambda_{\mathrm{H\alpha}}-1$. It is evident that the number of successful detections is suppressed near OH contamination. We take into account these effects on the radial distribution of the target galaxies, as described in Section 3.2.

2.4. Mock catalog

We validate our correction schemes (described in Section 4) for observational biases by using a set of mock samples constructed from cosmological $N$-body simulations. We utilize the new numerical galaxy catalog ($\nu^{2}$GC, Ishiyama et al. 2015) to take advantage of its large simulation volume. We use the medium-volume simulation ($\nu^{2}$GC-M) that was conducted by using $4096^{3}$ particles with a mass resolution of $2.2\times 10^{8}~{}h^{-1}~{}M_{\odot}$ in a comoving box with a side length of $560~{}h^{-1}~{}\mathrm{cMpc}$. We employ the halo catalog at a scale factor $a=0.384871$ ($z=1.598$), close to our median redshift ($z=1.588$), in which halos and subhalos are identified with the Rockstar algorithm (Behroozi et al., 2013b). For our purpose, the simulation box is divided into 64 sub-volumes of $70\times 70\times 560~{}(h^{-3}~{}\mathrm{cMpc})^{3}$, each of which can encloses the entire FMOS survey volume. In each sub-box, we mimic our observations to construct realistic mock target catalogs. By doing so, the spatial distribution of mock galaxies reflects the artificial biases due to the fiber allocation and inhomogeneous detection, similarly to the real data. We use them to establish and examine the correction schemes for these biases. Details of the construction of the mock catalogs and the correction schemes are described in Appendix A.

Furthermore, we assess the effect of stellar mass incompleteness. For this purpose, we use the Bolshoi simulation (Klypin et al., 2011) to take the advantage of its better mass resolution than $\nu^{2}$GC, and the availability of merger histories of halos. This simulation traces $2048^{3}$ particles in a cubic box with a side length of $250~{}h^{-1}~{}\mathrm{cMpc}$. We use the public catalog at $a=0.38435$ ($z=1.602$) to construct mock samples. We describe further the construction of the mock samples and the assessment of selection effects in Appendix B.

3. Clustering measurement

3.1. Two-point correlation function

The two-point auto-correlation function is a powerful and commonly-used tool to quantify the spatial distribution of galaxies. The real-space correlation function $\xi\left(r\right)$ measures the excess of the probability of finding pairs of galaxies as a function of their separation $r$. As a measure of the correlation function, we use the Landy & Szalay (1993) estimator:

[TABLE]

where $N_{D}$ and $N_{R}$ are the numbers of galaxies and random objects, respectively, $DD\left(r\right)$, $DR\left(r\right)$, $RR\left(r\right)$ are the numbers of data–data, data–random, random–random pairs with a comoving separation within the interval $\left[r,r+dr\right]$, respectively.

The radial distance computed from the redshift is different from the actual distance due to the peculiar motion of the galaxy, which results in the distortion of the correlation function measured in redshift space. To minimize these effects, we measure the correlation function on a two-dimensional grid, parallel and perpendicular to the line of sight, and integrate along the line-of-sight direction as standard practice for clustering analysis. We define the separation of objects following Fisher et al. (1994). Given a pair of objects at positions $\vec{r}_{1}$ and $\vec{r}_{2}$ in the redshift comoving space, the separation $\vec{s}$ and the line-of-sight vector $\vec{l}$ are defined as $\vec{s}=\vec{r}_{1}-\vec{r}_{2}$ and $\vec{l}=\left(\vec{r}_{1}+\vec{r}_{2}\right)/2$, respectively. The parallel ($\pi$) and perpendicular ($r_{\mathrm{p}}$) separations are given, respectively, as

[TABLE]

The projected correlation function $w_{\mathrm{p}}\left(r_{\mathrm{p}}\right)$ is related to the real-space correlation function as follows:

[TABLE]

If $\pi_{\mathrm{max}}$ is infinity, the redshift space distortions have no effect on $w_{\mathrm{p}}(r_{\mathrm{p}})$. In practice, it should be large enough to eliminate the effect of peculiar motions, but finite to avoid adding noise to the measurements. We adopt $\pi_{\mathrm{max}}=30~{}h^{-1}~{}\mathrm{cMpc}$, which corresponds to $\Delta z\approx\pm 0.024$ or $\Delta v\approx\pm 2800~{}\mathrm{km~{}s^{-1}}$. We count pairs of galaxies and random objects on the two-dimensional grid that is binned logarithmically in the $r_{\mathrm{p}}$ direction and linearly in $\pi$. Considering the small number of galaxy pairs with a small separation ($r_{\mathrm{p}}\lesssim 1~{}h^{-1}~{}\mathrm{cMpc}$), we employ a set of variable-size bins that have larger widths at small scales.

3.2. Construction of random samples

To measure galaxy clustering with the estimator given in Equation 2, we need to construct a reference random sample that follows the same geometrical properties as the real data. To avoid introducing shot noise, the random sample contains a large number of objects ($N_{R}=45000$ for our case), which is about 90 times larger than the spec-$z$ sample. Random objects are distributed uniformly across the effective survey area, while we consider a non-uniform radial distribution to reflect the realistic redshift distribution of our sample.

Determination of the radial distribution of observed galaxies to create the random sample represents a significant challenge. The simplest way is to randomly assign objects the same redshifts as the sample galaxies. However, this method is useful for only wide-field surveys that are not impacted strongly by cosmic variance. Given the survey area, our spectroscopic redshifts may suffer from this effect as the measured redshift distribution may reflect specific structures in the galaxy distribution. It can lead to an artificial line-of-sight clustering in the random catalog, and, subsequently, an underestimate of the correlation strength of real galaxies. In fact, with a reference catalog constructed in this way, we find a smaller correlation length by $\sim 0.6~{}h^{-1}~{}\mathrm{cMpc}$ than our final result obtained with the random catalog constructed as described below.

To define the underlying redshift distribution of galaxies, we use $\sim 10^{4}$ galaxies with $K_{\mathrm{S}}\leq 23.5$ and $M_{\ast}\geq M_{\ast}^{\mathrm{lim}}$ in the COSMOS photometric catalog (Ilbert et al., 2013). Here, no limitation on the photometric redshift is applied. Figure 6a shows the distribution of the radial comoving distance calculated from the photometric redshift of those galaxies. We smooth the binned distribution (histogram) with a Gaussian kernel with a standard deviation of 150, 250, or 450 $h^{-1}~{}\mathrm{cMpc}$. The distribution smoothed with the shortest kernel ($150~{}h^{-1}~{}\mathrm{cMpc}$) still traces specific structures (red line in Figure 6a). In contrast, smoothing with the longest kernel ($450~{}h^{-1}~{}\mathrm{cMpc}$) may produce a slight artificial enhancement at both ends and suppression around the peak of the distribution (blue dashed line). Therefore, we decided to use the distribution smoothed with an intermediate scale of $250~{}h^{-1}~{}\mathrm{cMpc}$ (green dotted line), which traces well the global shape of the distribution. We then estimate the intrinsic distribution of true redshifts of the FMOS-parent sample by taking into account the uncertainties on the photometric redshifts. Figure 6b illustrates our method. We extract the smoothed distribution between $1.46\leq z_{\mathrm{phot}}\leq 1.72$ (dotted line), and convolve it with a Gaussian function having a standard deviation $\sigma_{z_{\mathrm{phot}}}=0.062$, which is a typical error on the photometric redshift of our sample galaxies. As a result, we obtain the realistic radial distribution of our parent sample (dashed line).

We also consider the effects of the OH airglow mask and the inhomogeneous sensitivity. The noise level of pixels impacted by the sky contamination is much higher than the typical level of $\sim 5\times 10^{-19}~{}\mathrm{erg~{}cm^{-2}~{}s^{-1}\textrm{\AA}^{-1}}$ (see Figure 11 in Silverman et al. 2015) thus these pixels are ignored in the emission-line fitting. As a consequence, the detection rate decreases at the positions around the OH lines (Figure 5). We assess the detection rate of the H$\alpha$ emission line as a function of redshift over the FMOS $H$-long spectral window by performing a set of Monte-Carlo simulations. For each of 516 galaxies in the FMOS-spec-$z$ sample, we create artificial spectra with a multi-Gaussian profile that has the measured amplitude and line width for H$\alpha$ and [Nii]$\lambda\lambda$6548,6583. Gaussian noise is added to these artificial spectra based on the noise spectrum of each galaxy. Here the noise level is intentionally increased by a factor of 1.25 to account for the fact that the noise level tends to be underestimated relative to the actual pixel variance (see Silverman et al. 2015). We then perform a fitting procedure for these artificial spectra in the same manner as the data and examine if the H$\alpha$ line is recovered. We assess the detectability of this artificial signal at all pixels in the $H$-long window by scanning the entire spectral range. We accept as a success the cases for which the line is detected with $S/N\geq 3$ and the difference between the input and measured redshifts $\Delta z/(1+z)<0.001$. Figure 7 shows the average detection rate with positions of OH lines marked. As evident, the decrease of the detection rate is seen at the positions of the OH lines. In addition, the detectability falls down at both ends of the spectral coverage where the noise level becomes relatively high due to the instrumental characteristics. These features are expected to introduce artificial clustering in the line-of-site direction. This can be cancelled by using the random catalog that includes the same features. We take into account these instrumental effects on the radial selection function in the random catalog by multiplying the intrinsic redshift distribution derived above based on the photometric catalog (dashed curve in Figure 6) by this weight function shown in Figure 7. The constructed final weight function is given in Figure 8 in comparison with the distribution of observed spectroscopic redshifts. The global trend of the data is well represented by the weight function, while some spikes in the histogram may reflect the intrinsic clustered structures of galaxies. We demonstrate the impact of this inhomogeneous selection function and the validity of our correction scheme in Appendix A.

3.3. Statistical error estimates

We estimate the statistical errors of the observed correlation function using a jackknife resampling method. We construct twenty jackknife samples of the FMOS-spec-$z$ sample as follows. We first divide the entire survey area into four contiguous subregions of equal area (see Figure 1). The survey volume is then divided into five slices along the line of sight. Each sub-volume has a typical transverse side length of $20~{}h^{-1}~{}\mathrm{cMpc}$ and a depth of $80~{}h^{-1}~{}\mathrm{cMpc}$. For each jackknife sample, a sub-volume (1/20 of the full volume) is omitted in turn.

Since each spatial structure of galaxies affects the pair counts at different separations, the values of $w_{\mathrm{p}}(r_{\mathrm{p}})$ at different $r_{\mathrm{p}}$’s are correlated. Therefore, we need to use the full covariance matrix to fit a model to the data. The associated covariance matrix $\mathbb{C}_{ij}$ is estimated from the jackknife samples as follows:

[TABLE]

where $N=20$ is the number of jackknife samples, $X^{k}_{i}$ is the projected correlation function at the $i$-th separation measured for the $k$-th jackknife sample, and $\left<X_{i}\right>$ is the average of $X^{k}_{i}$ from $k=1$ to $N$. Here we use the logarithmic values $\log w_{\mathrm{p}}(r_{\mathrm{p}})$ rather than $w_{\mathrm{p}}(r_{\mathrm{p}})$ following the suggestion of Norberg et al. (2009). The choice of logarithmic or linear value does not change our conclusions.

Each element of the covariance matrix has large uncertainties due to the small number of jackknife samples. Therefore, we smooth the covariance matrix separately for the diagonal or off-diagonal elements following Mandelbaum et al. (2013). The diagonal elements (i.e., variance) are smoothed by a center-weighted kernel as follows:

[TABLE]

Figure 9 (top panel) shows the jackknife covariance diagonal elements and the smoothed values. The discontinuities at $r_{\mathrm{p}}\approx 0.15$ and $6~{}h^{-1}~{}\mathrm{cMpc}$ are effectively mitigated while the global shape is preserved. To smooth the off-diagonal elements, we first define the correlation matrix $\mathbb{R}_{ij}=\mathbb{C}_{ij}/\sqrt{\mathbb{C}_{ii}\mathbb{C}_{jj}}$, then we smooth it to obtain $\mathbb{R}^{\mathrm{smooth}}_{ij}$ with a $3\times 3$ kernel as follows:

[TABLE]

The final smoothed covariance matrix $\mathbb{C}^{\mathrm{smooth}}_{ij}$ is calculated as follows,

[TABLE]

The original and smoothed correlation matrices are shown in Figure 9 (middle and bottom panels). The majority of pixel-to-pixel fluctuations are well eliminated by smoothing while the overall trend is retained. We finally note that our conclusion does not depend on whether using the full covariance matrix or only the diagonal elements, or details of the smoothing method.

3.4. Model fit

To obtain physical insight, we fit a power-law model, a biased dark matter model, and an HOD model to the observed correlation function. We use 11 data points at $-1.5<\log(r_{\mathrm{p}}/h^{-1}~{}\mathrm{cMpc})<1.1$ for the model fitting. At larger scales ($r_{\mathrm{p}}\gtrsim 15~{}h^{-1}~{}\mathrm{cMpc}$), the contribution from the integral constraint becomes greater than 10% of the measured clustering amplitude (see Section 4.2).

We define the posterior distribution of our model parameters to be given by

[TABLE]

Here, $\chi_{\mathrm{data}}^{2}$ is calculated from the observed correlation function as

[TABLE]

where $X^{\mathrm{mod}}_{i}$ and $X^{\mathrm{obs}}_{i}$ are logarithms of the model and observed correlation functions at the $i$-the separation, $N$ is the number of data points, and $(\mathbb{C}^{\mathrm{smooth}}_{ij})^{-1}$ is the inverse of the smoothed covariance matrix defined in Section 3.3. To compare with the measurements based on pair counting in bins of $r_{\mathrm{p}}$, we calculate the average of the model $\log w_{\mathrm{p}}(r_{\mathrm{p}})$ in each $r_{\mathrm{p}}$ bin [$r_{\mathrm{min}}:r_{\mathrm{max}}$] as follows,

[TABLE]

In Equation (9), the latter term $\chi_{\mathrm{prior}}^{2}$ is the contribution from the prior imposed on the parameters. To sample the posterior distribution of our parameters, we adopt a Markov Chain Monte Carlo (MCMC) technique using the software emcee (Foreman-Mackey et al., 2013). We analyze a chain of 152400 steps that follows 25400 burn-in steps to find the parameter set that provides the maximum posterior probability, and to evaluate the posterior probability distribution of each parameter.

4. Corrections for critical biases

In spectroscopic galaxy surveys, various observational effects can cause non-negligible, artificial biases. As described in Section 3.2, we deal with the effects of the non-uniform detection of the H$\alpha$ emission line along the redshift direction by using a modified random sample (see also Appendix A). Here we describe our treatment for other systematic effects.

4.1. Fiber allocation

The most important issue is the impact of fiber allocation that artificially distorts the on-sky distribution of objects, if not all galaxies in the input catalog are observed. For our FMOS observations, galaxies are selected from an input catalog by using the Echidna Spine-to-Object allocation software (Akiyama et al., 2008) to maximize the operational efficiency. The FMOS fibers are uniformly embedded in the field-of-view, and they can move within a limited circular patrol area of $174~{}\mathrm{arcsec}$ in diameter. Once a pair of fibers is allocated for one galaxy, the opportunity for its neighboring galaxies to be observed at the same time decreases due to the lack of fibers and/or the avoidance of fiber entanglement, although the patrol areas of adjacent fibers overlap with each other. As a consequence, the sampling rate of close galaxy pairs is suppressed at scales less or similar to the minimum separation of fibers ($\sim 1.6^{\prime}$). In addition, the sampling rate is different across the survey area due to different number of repeated exposures covering the same footprint and the overlapping regions. These characteristics of our observations affect the observed correlation function of galaxies and thus need to be properly removed to measure the galaxy clustering.

We correct the observed correlation function for these biases by using a simple weighting scheme, in which each galaxy-galaxy pair is weighted in response to their angular separation (de la Torre et al., 2011; Durkalec et al., 2015a). The weight is defined as a ratio of the probability of finding pairs with a given angular separation in the input catalog to the sample of galaxies that were assigned to be observed. The weight function can be expressed by the angular correlation function of these two samples ($\omega_{\mathrm{par}}\left(\theta\right)$ and $\omega_{\mathrm{tar}}\left(\theta\right)$, respectively) as follows:

[TABLE]

It may be straightforward to determine the weight function based on the real data (i.e., the FMOS-parent and FMOS-fiber-target samples). However, the statistical uncertainties in the resulting weight function becomes large because the sample size is not sufficiently large. Therefore, we decided to use the mock samples to avoid such large statistical uncertainties. In Appendix A, we fully describe the construction of the mock samples and the determination of the weight function, and demonstrate the validity of our correction scheme.

4.2. Integral constraint

Due to the finite survey area, the observed correlation function is underestimated by a scale-independent constant value $C$, which is known as the integral constraint:

[TABLE]

First we calculate the integral constraint for the real-space correlation function $\xi(r)$ following Roche et al. (1999) as

[TABLE]

where $RR\left(r_{i}\right)$ is the number of random-random pairs whose separation is in the linearly-spaced $i$-th bin (bin size $\Delta r=1~{}h^{-1}~{}\mathrm{cMpc}$). The summation is taken over the entire survey volume. The relation between $C$ and $C_{\xi}$ is simply given by

[TABLE]

We use a biased non-linear correlation function of dark matter as the model function, i.e, $\xi^{\mathrm{mod}}=b^{2}\xi_{\mathrm{dm}}$. We first evaluate $b^{2}$ by comparing the non-linear projected correlation function of dark matter with the observed $w^{\mathrm{obs}}_{\mathrm{p}}(r_{\mathrm{p}})$, and calculate the integral constraint using Equations 14 and 15. We then revise $b^{2}$, and recalculate $C$. We find $C=0.80~{}h^{-1}~{}\mathrm{cMpc}$ for our data by repeating this process until convergence. This value is comparable to the observed correlation amplitude at scales greater than $r_{\mathrm{p}}\sim 20~{}h^{-1}~{}\mathrm{cMpc}$. Therefore, we use only measurements at scales smaller than this for analyses.

4.3. Contamination by fake detection

We correct the observed correlation function for contamination by fake sources for which a spurious signal or non-H$\alpha$ emission line is misidentified as H$\alpha$. Such contamination is expected to reduce the amplitude of the observed correlation function since these sources are not correlated with other real galaxies. Assuming that such sources are randomly distributed over the survey volume, the correlation function needs to be corrected by increasing its amplitude by a factor $1/(1-f_{\mathrm{fake}})^{2}$ where $f_{\mathrm{fake}}$ is the fraction of fake sources in the sample.

We assess the reliability of our redshift measurements using an independent spectroscopic survey that includes the same galaxies. In the FMOS-spec-$z$ sample, we find 28 galaxies that have a robust redshift measurement (confidence class 3 or 4) from the zCOSMOS-deep survey (Lilly et al., 2007). Of these, 24 galaxies have the redshift measurements from the FMOS-COSMOS and zCOSMOS surveys that are mutually consistent ($\Delta z<0.01$), while four other objects have inconsistent measurements. Assuming that our FMOS measurements are wrong for these objects and all 28 zCOSMOS redshifts are correct, we find a plausible fraction of fake detection to be $f_{\mathrm{fake}}=0.14$. This fraction corresponds to an underestimate of the correlation function by 26%. We note that the four objects with a possible wrong measurement are not either revised or removed from the sample for analysis, although one of them match to the zCOSMOS measurement if we suppose that the [Oiii]$\lambda 5007$ is misidentified as H$\alpha$.

In the fitting process, we use the observed correlation function without the correction for the contamination from fake detections. Instead, we include this effect in our analyses by handling $f_{\mathrm{fake}}$ as a parameter to be estimated together with other model parameters. Namely, we compare the model multiplied by $(1-f_{\mathrm{fake}})^{2}$ to the data. Assuming Poisson statistics, 4/28 approximately translates to $0.14\pm 0.06$, which is imposed as a prior probability distribution on $f_{\mathrm{fake}}$. However, the zCOSMOS-deep survey is targeting rather higher redshift galaxies at $z>2$, while having a less sampling rate at the redshift range of our FMOS sample. Therefore, the fake fraction may be overestimated. Considering the large uncertainty in the estimation of $f_{\mathrm{fake}}$, we report results obtained by using this prior on $f_{\mathrm{fake}}$ as fiducial, while presenting results with $f_{\mathrm{fake}}$ fixed to zero for reference. In all cases, our results for these two cases are consistent at a $1\sigma$ level. We note that this prior dominates the posterior probability distribution of $f_{\mathrm{fake}}$ for the power-law and biased dark matter models (see Sections 5.1 and 5.2, respectively) because there is no other information that can constrain the amplitude of the correlation function. This prior on $f_{\mathrm{fake}}$ broadens the posterior distribution of the parameter that determines the correlation amplitude, i.e., correlation length or galaxy bias. In contrast, in the HOD modeling (Section 6.1), the posterior of $f_{\mathrm{fake}}$ is modified from the prior as the amplitude and shape of correlation function, and the total galaxy abundance are linked through the model.

5. Results

The projected correlation function $w_{\mathrm{p}}(r_{\mathrm{p}})$ is computed for a sample of 516 star-forming galaxies at $1.43\leq z\leq 1.74$ in the central $0.81~{}\mathrm{deg}^{2}$ (effectively $0.77~{}\mathrm{deg}^{2}$) of the COSMOS field. Figure 10 presents the observed $w_{\mathrm{p}}(r_{\mathrm{p}})$ with and without the correction for the scale-dependent effect of fiber allocation. In both cases, the measurements are corrected for the integral constraint, but not for the effect of fake detections. Instead, the model functions are reduced by $(1-f_{\mathrm{fake}})^{2}$ where $f_{\mathrm{fake}}=0.14$. As evident, the amplitude of the correlation function without the correction for the fiber allocation effect is slightly suppressed as compared to the corrected values at small scales below $r_{\mathrm{p}}\sim 1~{}h^{-1}\mathrm{Mpc}$ (see Section 4.1 and Appendix A for details). We use the corrected values as the fiducial measurements throughout the paper. The error bars indicate the standard deviation of $\log w_{\mathrm{p}}(r_{\mathrm{p}})$ that is estimated from jackknife resampling (see Section 3.3).

5.1. Power-law model

It is known that a real space galaxy correlation function $\xi\left(r\right)$ can be well described by a power-law function as $\xi\left(r\right)=(r/r_{0})^{-\gamma}$(e.g., Totsuji & Kihara, 1969; Zehavi et al., 2002), where $r_{0}$ and $\gamma$ are a correlation length and power-law slope, respectively. The correlation length denotes how strongly galaxies are clustered. With this form of $\xi\left(r\right)$, from Equation (4), $w_{\mathrm{p}}\left(r_{\mathrm{p}}\right)$ can be expressed as

[TABLE]

where $\Gamma$ is Euler’s Gamma function and the integration limit $\pi_{\mathrm{max}}$ of Equation (4) is taken as infinity. We fit the form of Equation (16), multiplied by the contamination factor $(1-f_{\mathrm{fake}})^{2}$, to the observed $w_{\mathrm{p}}\left(r_{\mathrm{p}}\right)$ (see Figure 10), and search the parameter space ($r_{0}$, $\gamma$, $f_{\mathrm{fake}}$) with the MCMC procedure. We find a correlation length to be $r_{0}=5.21_{-0.67}^{+0.70}~{}h^{-1}~{}\mathrm{cMpc}$ with a slope $\gamma=1.99_{-0.17}^{+0.13}$ for our sample. The parameter constraints are shown in Figure 11. The best-fit parameters and the associated uncertainties (68% confidence intervals) are listed in Table 3.

5.2. Biased dark matter model

The galaxy distribution is biased relative to the underlying matter distribution because galaxies form at peaks of the dark matter density fluctuations. A correlation function of galaxies is related to that of dark matter $\xi_{\mathrm{dm}}(r)$ by the galaxy bias $b$ as,

[TABLE]

where we calculate $\xi_{\mathrm{dm}}$ from the non-linear matter power spectrum derived by Smith et al. (2003). For a scale-independent bias, Equation (17) can be simply rewritten for the projected form as

[TABLE]

where $w_{\mathrm{p,dm}}$ is the projection of $\xi_{\mathrm{dm}}$. We fit the $b^{2}w_{\mathrm{p,dm}}(r_{\mathrm{p}})$ to the 7–11th data points at $r_{\mathrm{p}}>1~{}\mathrm{h^{-1}cMpc}$ to avoid an enhancement of the resulting galaxy bias due to the significant one-halo term of the data. We find $b=2.59_{-0.34}^{+0.41}$ (see Table 3). The best-fit model and the parameter constraints are shown in Figures 10 and 12, respectively. We note that $b=2.32_{-0.22}^{+0.25}$ is obtained when only the errors on each data point are used to calculate $\chi^{2}$, rather than the full covariance matrix.

5.3. Comparisons of clustering strength

In Figure 13, we compare the observed correlation length ($r_{0}=5.21_{-0.67}^{+0.70}~{}h^{-1}~{}\mathrm{cMpc}$) with other previous measurements up to $z\sim 5$ and predictions from the dark halo model. Most past studies presented here used samples of star-forming galaxies with stellar masses or luminosity similar to our sample (typically $M_{\ast}\sim 10^{10}~{}M_{\odot}$ and $L_{\mathrm{H\alpha}}\sim 10^{42}~{}\mathrm{erg~{}s^{-1}}$), thus can be straightforwardly compared to each other.

At lower redshifts, Shioya et al. (2008) and Nakajima et al. (2008) measured clustering of narrow-band-selected H$\alpha$-emitters (HAEs) at $z\sim 0.24$ and $z\sim 0.4$, respectively. They found a correlation length to be $r_{0}=1.3~{}h^{-1}\mathrm{cMpc}$ ($z\sim 0.24$) and $r_{0}=1.1~{}h^{-1}\mathrm{cMpc}$ ($z\sim 0.4$), which are smaller than our results and others (filled triangles in Figure 13). Nakajima et al. (2008) mentioned that the low H$\alpha$ luminosity limits of their samples ($L_{\mathrm{H\alpha}}\geq 10^{39.8}~{}\mathrm{erg~{}s^{-1}}$), which is lower by roughly two orders of magnitude than that of our sample, and relatively large H$\alpha$ equivalent widths (i.e., their lower stellar masses) are likely responsible for the observed weak clustering strengths.

The HiZELS team conducted a wide-field, near-infrared narrow-band survey of HAEs. Sobral et al. (2010) presented a clustering analysis of HAEs at $z\sim 0.84$ and found $r_{0}\sim 2.7~{}h^{-1}\mathrm{cMpc}$ and $\sim 4.8~{}h^{-1}\mathrm{cMpc}$ for their entire sample and a brighter subsample ($L_{\mathrm{H\alpha}}\geq 10^{42}~{}\mathrm{erg~{}s^{-1}}$), respectively. The latter measurement is in rough agreement with our result, as expected from the similar H$\alpha$ luminosity limit. In addition, Geach et al. (2012) measured a correlation function at $z\sim 2.23$ with $\mathrm{SFR}\gtrsim 7~{}M_{\odot}~{}\mathrm{yr^{-1}}$, and found $r_{0}\sim 3.7~{}h^{-1}~{}\mathrm{cMpc}$. This slightly smaller correlation length (than ours and the HiZELS result at $z=0.84$) may be reasonable given the lower SFR limit for their sample.

There have been many studies that carry out a clustering analysis for color-selected star-forming galaxies. Adelberger et al. (2005) measured the clustering for BM ($z\sim 1.7$) and BX ($z\sim 2.2$) galaxies (Adelberger et al., 2004) with $r_{0}=4.5~{}h^{-1}\mathrm{cMpc}$ and $4.2~{}h^{-1}\mathrm{cMpc}$. Hartley et al. (2010) presented a measurement for rest-frame $U-V$ selected star-forming galaxies and found that the correlation length of subsamples at a fixed $K$-band luminosity (a proxy of stellar mass) decreases since $z\sim 3$ to the present day. Their measurements at $z\sim 1.6$ are similar to ours. Bielby et al. (2014) used a sample of $NUV-r$ selected star-forming galaxies at $z\sim 0.5$–1.75, showing $r_{0}\sim 4$–$5~{}h^{-1}~{}\mathrm{cMpc}$ for galaxies with $\log M_{\ast}/M_{\odot}=9.57$–$11$. Ishikawa et al. (2015) found $r_{0}=4.12\pm 0.07~{}h^{-1}~{}\mathrm{cMpc}$ for $gzK$-selected star-forming galaxies with $K<23$. The $BzK$ color selection is known to well select star-forming (sBzK) galaxies at $1.4\lesssim z\lesssim 2.5$ (Daddi et al., 2004). Hayashi et al. (2007) found $r_{0}=3.2~{}h^{-1}~{}\mathrm{cMpc}$ for a sample of sBzK galaxies with $K<23.3$ and $\left<z\right>=1.9$. Hartley et al. (2008) measured a stronger clustering strength with $r_{0}\sim 6.8~{}h^{-1}~{}\mathrm{cMpc}$ and argued that contamination from highly clustered galaxies at higher redshifts ($z\gtrsim 2.5$) may be responsible for such a high amplitude. Lin et al. (2012) found $r_{0}\sim 6.3~{}h^{-1}~{}\mathrm{cMpc}$ for sBzK galaxies of $10<\log(M_{\ast}/M_{\odot})<10.5$. At $z\sim 1.6$–$1.7$, McCracken et al. (2010) measured an angular correlation function of $K_{\mathrm{s}}<23~{}(22)$ sBzK galaxies in the COSMOS field with $r_{0}\sim 4.25~{}(4.69)~{}h^{-1}~{}\mathrm{cMpc}$. Another study in COSMOS (Béthermin et al., 2014) found $r_{0}=4.0\pm 0.9$ for sBzK galaxies of $M_{\ast}\sim 10^{10}M_{\odot}$. While these measurements are slightly different from one study to another, our measurement is in rough agreement and bracketed by measurements from past studies using a sample with a range of stellar mass or luminosity similar to our sample.

At higher redshifts, there have been efforts to measure the clustering of Lyman break galaxies (LBGs) at $z\sim 3$–5 (e.g., Bielby et al., 2013; Kashikawa et al., 2006; Foucaud et al., 2003; Adelberger et al., 2005) or beyond (Harikane et al., 2016), finding correlation lengths to be $r_{0}\sim 4$–$5~{}h^{-1}~{}\mathrm{cMpc}$. Durkalec et al. (2015a) measured the projected correlation function of $\sim 3000$ galaxies from the VIMOS Ultra Deep Survey, including the COSMOS field, and found $r_{0}\sim 4~{}h^{-1}\mathrm{cMpc}$ at $z\sim 2.5$–$5$. These measurements are similar to those for star-forming galaxies at $z\sim 1\textrm{--}2$, including our own, although the average properties of LBGs may not be identical to lower-redshift color- or H$\alpha$-selected star-forming galaxies.

Given these comparisons, we conclude that our measurement is in general agreement with other previous measurements for star-forming galaxies at similar redshifts. In Figure 13, no clear evolutionary trend can be seen since $z\lesssim 4$. This is consistent with the moderate evolution of $r_{0}$ for a single population of galaxies predicted by the $\Lambda$CDM framework (e.g., Kauffmann et al., 1999). However, it is generally difficult to compare the clustering strengths at different redshifts because measurements are based on different selection functions.

6. Connection between galaxies and dark matter halos

Knowledge of the connection between galaxies and dark matter halos is essential to understand how and in what environments galaxy form and evolve. We investigate the properties of halos of “main-sequence” star-forming galaxies with $M_{\ast}\gtrsim 10^{9.57}~{}M_{\odot}$ at $z\sim 1.6$ by interpreting the observed correlation function with an HOD model.

6.1. The halo model and occupation distribution

In the standard CDM paradigm, dark matter halos form at peaks in the matter density field. The global properties of halos such as their abundance (or the halo mass function) and large scale clustering amplitude (or the halo bias) are primarily determined by halo mass (Press & Schechter, 1974; Cole & Kaiser, 1989; Mo & White, 2002). Galaxies reside in dark matter halos. Therefore, the observable abundance and clustering of galaxies can be used to constrain the connection between galaxies and dark matter halos.

The HOD framework is a convenient parametric way to describe the galaxy–dark matter connection to model the abundance and clustering of galaxies. In its simplest form, the HOD describes the average number of galaxies, $\langle N|M,z\rangle$, that reside in a halo of mass $M$ at redshift $z$, and assumes that this number does not depend upon the formation history and the environment of halos. For the model implemented here, we subdivide galaxies into being either central or satellite galaxies (e.g., Zheng et al., 2005), depending upon their location within their dark matter halos, such that

[TABLE]

The average number density of galaxies is then simply given by

[TABLE]

where the halo mass function, $n(M,z){\rm d}M$, gives the number density of halos of mass $M\pm{\rm d}M/2$ at redshift $z$. The average number density of central ($n_{\mathrm{cen}}$) or satellite galaxies ($n_{\mathrm{sat}}$) is calculated by replacing $\left<N|M,z\right>$ in Equation (20) with $\left<N_{\mathrm{cen}}|M,z\right>$ or $\left<N_{\mathrm{sat}}|M,z\right>$. For calculations, we fix the redshift to the median of the sample ($z=1.588$). Hereafter, the variable $z$ is omitted from equations.

The clustering of galaxies can be quantified as the excess probability over the random case of finding two galaxies separated by a distance and is described by the two-point correlation function $\xi$. The clustering of galaxies arises from a combination of two contributions. The one-halo term corresponds to the pairs of galaxies that reside within the same halo and the two-halo term to the pairs of galaxies which reside in distinct halos:

[TABLE]

where the superscripts “1h” and “2h” stand for the one-halo and the two-halo terms, respectively. The correlation function $\xi(r)$ and the power spectrum $P(k)$ form a Fourier transform pair such that

[TABLE]

which implies that Equation (21) can also be written as

[TABLE]

The one-halo term can be further expressed as the sum of contributions from the central-satellite and satellite-satellite galaxy pairs hosted by the same halo:

[TABLE]

where $u(k,M)$ describes the Fourier transform of the density profile of satellite galaxies within dark matter halos. We assume that central galaxies reside at the center of halos and that the occupation numbers of centrals and satellites are independent of each other, so that $\left<N_{\mathrm{cen}}N_{\mathrm{sat}}\right>=\left<N_{\mathrm{cen}}\right>\left<N_{\mathrm{sat}}\right>$. We have further assumed that $N_{\mathrm{sat}}$ follows Poisson statistics, so that $\left<N_{\mathrm{sat}}(N_{\mathrm{sat}}-1)\right>=\left<N_{\mathrm{sat}}\right>^{2}$. This is supported both by observations (Yang et al., 2008) and numerical simulations (Kravtsov et al., 2004).

The two-halo term consists of contributions from the central–central, central–satellite, and satellite–satellite galaxy pairs hosted by distinct halos:

[TABLE]

where $P_{\mathrm{hh}}(k|M_{1},M_{2})$ describes the cross power spectrum of halos of masses $M_{1}$ and $M_{2}$. Following van den Bosch et al. (2013), we express it as a product of the large scale bias of halos of masses $M_{1}$ and $M_{2}$, and the non-linear matter power spectrum (Smith et al., 2003), and account for the radial dependence of the bias as well as for halo exclusion.

We assume that the HOD of central galaxies is given by

[TABLE]

and that of satellite galaxies is given by

[TABLE]

For $M<M_{\mathrm{cut}}$, $\left<N_{\mathrm{sat}}|M\right>=0$. The parameter $M_{\mathrm{min}}$ is a halo mass above which a halo has a central galaxy. This transition is relaxed with a smoothing scale $\sigma_{\log M}$ (Zheng et al., 2005, 2007). The average number of satellite galaxies increases with increasing halo mass by a power-law parametrized by a slope $\alpha$ and normalization $M_{1}^{\prime}$, which is related to the halo mass ($M_{1}$) at which a halo is expected to have a single satellite galaxy, as $M_{1}=M_{1}^{\prime}+M_{\mathrm{cut}}$.

For computing all observables, we use the halo mass function from Tinker et al. (2010), which defines a halo as a spherically-collapsed region with an average density 200 times greater than the background matter density of the Universe, and a large-scale halo bias proposed by Tinker et al. (2010) with an empirical radial scale dependence derived by Tinker et al. (2012) with corrections described in van den Bosch et al. (2013). In addition, we assume that the radial distribution of satellite galaxies follows the density distribution of dark matter in halos. We use the Navarro-Frenk-White profile (Navarro et al., 1997) with the mass-concentration relation calibrated by Macciò et al. (2007) for this purpose.

Once a set of HOD parameters is given, the following physical quantities are inferred:

• •

Effective large scale bias

[TABLE]

• •

Satellite fraction

[TABLE]

• •

Effective halo mass

[TABLE]

The effective large scale bias is the number-weighted average of the halo bias $b_{h}(M)$. The effective halo mass is the number-weighted average mass of halos that host galaxies in the sample.

6.2. Limitations for the HOD parameters

We sample the posterior distribution of the HOD parameters given the abundance and clustering measurements using a MCMC technique. While the HOD model defined above has five parameters, our data do not have enough statistics to constrain all the parameters simultaneously. We here describe the prior limitations imposed on some model parameters to resolve degeneracies and to avoid overfitting.

We fix the power-law slope $\alpha$ in Equation (30) to be 1, which is supported observationally (Zheng et al., 2005) and theoretically (Kravtsov et al., 2004), and has been commonly applied in past studies (e.g., Conroy et al., 2006). We further impose a relation between $M_{\mathrm{cut}}$ and $M_{1}^{\prime}$ as

[TABLE]

following Conroy et al. (2006). In addition, we impose priors on $\sigma_{\log M}$ independently by using a stellar-to-halo mass relation derived by Behroozi et al. (2013a) with the uncertainties on the stellar mass estimate taken into account (see Appendix C for details). The prior probability distribution is given by

[TABLE]

The prior on the contamination fraction $f_{\mathrm{fake}}$ is given as $f_{\mathrm{fake}}=0.14\pm-0.06$ (see Section 4.3). We use uniform non-informative priors on the halo mass parameters $\log M_{\mathrm{min}}/(h^{-1}M_{\odot})$ and $\log M_{1}^{\prime}/(h^{-1}M_{\odot})$, in the range $[9,15]$.

With the HOD model parametrized by Equations (29–30), the number density of all galaxies with $M_{\ast}\geq M_{\ast}^{\mathrm{lim}}$, i.e., the $M_{\ast}$-selected sample, can be predicted using Equation (20), and compared with the observed number density of $9.29\times 10^{-3}~{}h^{-3}\mathrm{cMpc}^{-3}$ (see Table 2). We emphasize that this is not the number density of the FMOS-parent sample with additional selection based on $K_{\mathrm{S}}$ and predicted H$\alpha$ flux. In doing so, we have implicitly assumed that our FMOS-spec-$z$ sample is representative of the $M_{\ast}$-selected sample (see Section 2.3). We estimate the cosmic variance of the number density by using the subhalo mock catalog from $\nu^{2}$GC simulation and find approximately 10% fluctuation from one to another sub-box with the same volume as our survey. Therefore, we use the observed abundance to be $n_{\mathrm{tot}}=(9.29\pm 0.93)\times 10^{-3}~{}h^{-3}\mathrm{cMpc}^{-3}$ as an additional independent observable to be compared to the predictions in the MCMC procedure. The prior information is summarized in Table 3.

6.3. HOD model fit

We compare the projected correlation function computed from the HOD model parameters to the observed $w_{\mathrm{p}}(r_{\mathrm{p}})$. We have varied three HOD parameters ($\log M_{\mathrm{min}}$, $\sigma^{2}_{\log M}$, $\log M^{\prime}_{1}$), while using 11 data points at $-1.5<\log(r_{\mathrm{p}}/h^{-1}~{}\mathrm{cMpc})<1.1$. As a result, we have 10 degrees of freedom in this analysis:

[TABLE]

The three HOD parameters can be effectively constrained using our observables. The posterior distribution of the HOD and the parameters are shown in Figures 14 and 15, respectively. In addition, we have also calculated various physical quantities using Equations (20) and (31–33). The best-fit parameters and the inferred quantities are summarized in Table 3. The posterior probability distributions are shown in Figure 16. It can been seen that $M_{\mathrm{min}}$ is degenerate with $M_{1}^{\prime}$. This negative correlation is mainly caused by the prior constraint on the total abundance of galaxies because a smaller $M_{\mathrm{min}}$ leads to a more abundant number of centrals and thus $M_{1}^{\prime}$ needs to increase to reduce the number of satellites and vice versa. It also seems that the data prefers a lower value for the fraction of fake detections $f_{\mathrm{fake}}$ compared to the prior (shown by dotted line in Figure 15). We note that the results for $f_{\mathrm{fake}}=0$ are in agreement with the fiducial analysis within 1$\sigma$ (the rightmost column in Table 3). For the remainder of the paper, we use the results from the fiducial prior on $f_{\rm fake}$.

In Figure 17, we show the observed $w_{\mathrm{p}}(r_{\mathrm{p}})$ and the model computed from the best-fitting HOD parameters. The observed $w_{\mathrm{p}}(r_{\mathrm{p}})$ is corrected for the effect of fake detections with the best-fit value of $f_{\mathrm{fake}}=0.099$. The data points are well fit with the model with a clear signature of the one-halo term. We find that the transition from the one-halo-dominated regime at small scales to the two-halo-dominated regime at large scales happens at $r_{\mathrm{p}}\approx 0.5~{}h^{-1}~{}\mathrm{cMpc}$. Since the two-halo term dominates at scales larger than the typical virial radius of largest halos, this scale is a useful probe of the virial size of halos that host the galaxies. For SDSS galaxies ($z\lesssim 0.1$), such a transition happens around $r_{\mathrm{p}}=1\textrm{--}2~{}h^{-1}~{}\mathrm{cMpc}$ (Zehavi et al., 2004). In contrast, Zheng et al. (2007) measured the one-to-two halo transition at $r_{\mathrm{p}}\sim 0.4\textrm{--}0.6~{}h^{-1}~{}\mathrm{cMpc}$ at $z\sim 1$, which is in good agreement with our result. In addition, Gobat et al. (2015) found an excess of the number of satellite galaxies up to $\sim 300~{}\mathrm{physical~{}kpc}$ away from the halo centre at a similar redshifts ($z\sim 1.8$) through image stacking. This putative halo radius based on an independent technique is fully consistent with our measurement. These results show that the transition scale decreases for high redshift galaxies and this trend likely reflects the fact that massive clusters have less formed yet at earlier epochs.

7. Discussion

In this section, we discuss the properties of halos that contain star-forming galaxies at $z\sim 1.6$ based on the physical parameters inferred from the HOD modeling of the observed correlation function, and then present a new constraint on the stellar-to-halo mass relation. At the end, we demonstrate the capabilities of a future survey with a next-generation multi-object spectrograph.

7.1. Halo mass and large-scale bias

We find the effective halo mass to be $M_{\mathrm{eff}}=4.62_{-1.63}^{+1.14}\times 10^{12}~{}h^{-1}~{}M_{\odot}$ (Equation 33) and the effective large-scale galaxy bias to be $b_{\mathrm{eff}}=2.17_{-0.24}^{+0.17}$ (Equation 31). In Figure 18, we present $M_{\mathrm{eff}}$ in comparison to the average growth histories of halos with different present-day masses, as derived by Zhao et al. (2009) and Behroozi et al. (2013a). We find that $M_{\mathrm{eff}}$ of our sample lies on the mass assembly history of halos having a present-day mass $M_{\mathrm{h}}(z=0)\approx 2\times 10^{13}~{}h^{-1}M_{\odot}$ (thick gray curve). This is equivalent to the typical mass of group-scale halos. We also plot effective halo masses, derived through HOD modeling of galaxy clustering at different redshifts, from the literature. At lower redshifts, we present measurements from the CFHT Legacy Survey at $z\sim 0.3,0.5$ and 0.7 (Coupon et al., 2012). These samples are selected based on the absolute $g$-band magnitude ($M_{g}-5\log h<-19.8$) and the galaxy number densities are $n_{\mathrm{gal}}\approx 8\times 10^{-3}h^{3}\mathrm{cMpc^{-3}}$, similar to our $M_{\ast}$-selected sample. In addition, the results from Abbas et al. (2010) at $0.2\lesssim z\lesssim 1.3$ based on the VIMOS-VLT Deep Survey (VVDS; Le Fèvre et al. 2005) are shown. We highlight two subsamples in Abbas et al. (2010) with $M_{B}<-19.5$ at $z=0.67$ and 0.99 (filled triangles) that have a number density similar to our $M_{\ast}$-selected sample. We also present the results for samples with $M_{\ast}>10^{10}~{}M_{\odot}$ $z\sim 1.1$ and $\sim 1.5$ from the NEWFIRM Medium Band Survey (Wake et al., 2011). These samples at lower redshifts are in excellent agreement with the same halo mass assembly history as our data. At higher redshifts, we show results from HiZELS ($z\sim 2.2$; Geach et al. 2012) and VUDS ($z\sim 2.5$ and $\sim 3.5$; Durkalec et al. 2015a), which are in broad agreement as well, favoring a slightly lower mass ($M_{\mathrm{h}}\approx 10^{13}~{}h^{-1}M_{\odot}$ at $z=0$). In summary, these results indicate a good agreement between the predictions of the halo mass assembly history and observations over a wide range of cosmic history since $z\sim 4$ to the present. From another perspective, it is supported that these samples at different redshifts essentially represent similar galaxy populations at different epochs.

In Figure 19, we show the effective large-scale galaxy bias in comparison with measurements from the literature that are shown in Figure 18. Our data in broad agreement with measurements at $1\lesssim z\lesssim 2$ by Geach et al. (2012) and Wake et al. (2011). Based on measurements over a wide range of redshift, it seems that the galaxy bias decreases with cosmic time. This trend agrees with the general expectation from the standard scenario of hierarchical structure formation (e.g., Mo & White, 1996; Kauffmann et al., 1999). For reference, we show the halo bias for fixed halo masses ($\log M_{\mathrm{h}}/(h^{-1}M_{\odot})=10,~{}11,~{}12,~{}13$, and 14; thin solid lines), which present a rapid decline of the bias. As mentioned above, halos of a present-day mass $M_{\mathrm{h}}(z=0)=2\times 10^{13}~{}h^{-1}~{}M_{\odot}$ are expected to be the descendants of halos containing star-forming galaxies in our sample at $z\sim 1.6$ (thick gray line in Figure 18). We further calculate the halo bias for the evolving halo mass as a function of redshift. The global trend of the effective bias is well represented by this evolutionary track (thick gray line in Figure 19).

7.2. Stellar mass-to-halo mass ratio

The stellar mass-to-halo mass ($M_{\ast}/M_{\mathrm{h}}$) ratio encodes the efficiency of converting baryons into stars relative to the total amount of dark matter accreting onto halos. Yang et al. (2003) proposed a functional form with double power-law components to express the average halo mass-to-galactic luminosity ratios as a function of halo mass, motivated by the fact that the observed luminosity function is steeper (shallower) than the halo mass function at the high (low) mass end (see also Vale & Ostriker, 2004; Cacciato et al., 2013). With improvement of stellar mass measurements, this formalism has been applied to describe the stellar-to-halo mass relation (SHMR; $M_{\ast}/M_{\mathrm{h}}$ vs. $M_{\mathrm{h}}$) (e.g., Zheng et al., 2007; Yang et al., 2008, 2009). Past studies based on large data sets (e.g., SDSS) have shown that the $M_{\ast}/M_{\mathrm{h}}$ ratio reaches a peak around a halo mass of $M_{\mathrm{h}}\sim 10^{12}~{}M_{\odot}$ at $z\sim 0\text{--}1$ (e.g., Moster et al., 2010, 2013; Behroozi et al., 2010; Guo et al., 2010; More et al., 2011; Leauthaud et al., 2012; Coupon et al., 2015). Beyond $z>1$, there have been limited efforts to measure the SHMR based on HOD modeling of galaxy clustering (Wake et al., 2011; Geach et al., 2012; Durkalec et al., 2015a, b; Martinez-Manso et al., 2015; Harikane et al., 2016). Other studies use an alternative technique (i.e., abundance matching; Kravtsov et al. 2004) to predict the SHMRs (Moster et al., 2013; Behroozi et al., 2013a). It populates halos in a $N$-body simulation with galaxies while assuming an SHMR, and the model is adjusted to match the inferred galaxy abundance to observations.

For our sample of star-forming galaxies, we find $\log M_{\ast}/M_{\mathrm{h}}=-2.3\pm 0.2$ at $M_{\mathrm{h}}=M_{\mathrm{min}}=10^{11.86}~{}M_{\odot}$ (computed with $h=0.7$), which is determined as the ratio of the threshold stellar mass $M_{\ast}^{\mathrm{lim}}$ to $M_{\mathrm{min}}$. Figure 20 presents the observed $M_{\ast}/M_{\mathrm{h}}$ ratio as a function of $M_{\mathrm{h}}$. We emphasize that our result probes the low-mass side of the SHMR at this epoch based on a spectroscopic sample, evidently confirming the rapid decline at $M_{\mathrm{h}}<10^{12}~{}M_{\odot}$. We compare our result with past measurements at similar redshifts from the literature, which are based on HOD modeling of galaxy clustering using photometric redshifts and/or color-selected galaxy samples. Martinez-Manso et al. (2015) measure the $M_{\ast}/M_{\mathrm{h}}$ ratios over $12\lesssim\log M_{\mathrm{h}}/M_{\odot}\lesssim 13.2$ by using a sample of $z\sim 1.5$ galaxies selected from the Spitzer/IRAC 3.6 and 4.5 $\mathrm{\mu m}$ photometry. Their result shows an evident peak at $M_{\mathrm{h}}=10^{12.4}~{}M_{\odot}$. Around this peak halo mass, Wake et al. (2011) and McCracken et al. (2015) measure $M_{\ast}/M_{\mathrm{h}}$ at $z\sim 1.5$, which are systematically higher than our result. We note that the $M_{\ast}/M_{\mathrm{h}}$ ratios in Wake et al. (2011) are multiplied by a factor of 1.5 to account for the fact that the Maraston (2005) stellar population synthesis model induces stellar masses that are systematically lower than those derived by the Bruzual & Charlot (2003) model (on average by $\sim 60\%$; Maraston et al. 2006). The measurements by Wake et al. (2011) are in good agreement with the result of Martinez-Manso et al. (2015), whereas the measurements by McCracken et al. (2015) are systematically greater by a factor of $\sim 2$ at fixed $M_{\mathrm{h}}$. Another study (Gobat et al., 2015) compares the average stellar mass of a sample of massive star-forming galaxies at $z\sim 1.8$ ($\sim 1.3\times 10^{11}~{}M_{\odot}$) and the halo mass ($\sim 2.4\times 10^{13}~{}M_{\odot}$) induced from the average X-ray luminosity through stacking analysis. Their independent measurement clearly indicates, with others, the decline of SHMR at high masses. It may be worth noting that our measurement is in good agreement with the extrapolation of the SHMR derived by Martinez-Manso et al. (2015) (thin dashed line in Figure 20).

Furthermore, we show in Figure 20 the SHMRs at $z=1.6$ that are derived by an abundance matching technique (Behroozi et al., 2013a; Moster et al., 2013). The Behroozi et al. (2013a) SHMR presents a peak value that is higher (by $\sim 0.2~{}\mathrm{dex}$) than as in Martinez-Manso et al. (2015), while being more consistent with McCracken et al. (2015). In comparison among these studies, the three measurements based on the HOD modeling (our result, Martinez-Manso et al. (2015), and Wake et al. (2011)) likely favor an SHMR having the low-mass part that is systematically lower than that predicted by abundance matching. However, we note that our result is consistent with those at a $2\sigma$ confidence level, and becomes closer to the Behroozi et al. (2013a) SHMR (nearly $1\sigma$) when being recalculated with the same cosmology as the reference ($\Omega_{\mathrm{m}}=0.27$; dotted contours in Figure 20). For quantitative derivation and comparison of SHMRs, one needs to carefully treat the difference between adopted cosmologies (e.g., $\Omega_{\mathrm{m}},\sigma_{8}$) from one study to another. We refer the reader to More (2013) for further discussions of such systematic effects on the galaxy clustering.

7.3. Baryon conversion efficiency

Given an average growth history of halos and an observation of SFRs of galaxies, we can calculate the efficiency of baryon conversion $\epsilon_{\mathrm{baryon}}$, i.e., the fraction of the mass of baryons converted into stars per unit time to the total accretion rate of baryons falling into halos. For this purpose, the SFRs of our spec-$z$ galaxy sample are measured from the observed H$\alpha$ luminosities by using Equation (1). To derive intrinsic SFRs, we correct the observed H$\alpha$ flux for aperture loss (Silverman et al., 2015) and dust extinction assuming a Calzetti et al. (2000) reddening curve. The level of extinction $E_{\mathrm{star}}(B-V)$ is estimated based on the $B_{z}-j$ color (Daddi et al., 2007), and converted to the attenuation towards nebular emission lines assuming a relation $E_{\mathrm{neb}}(B-V)=E_{\mathrm{star}}(B-V)/0.66$ (see Kashino et al., 2013). We find the average SFR to be $\approx 15~{}M_{\odot}\mathrm{yr^{-1}}$ at $M_{\ast}^{\mathrm{lim}}=10^{9.57}~{}M_{\odot}$ with a standard deviation of $0.3~{}\mathrm{dex}$, including errors on the flux measurement. However, this average may be slightly biased toward a higher value as compared to the $M_{\ast}$-selected sample because of the imposed limit on the predicted H$\alpha$ flux and the detection bias (see Section 2.2). According to SED-based SFRs shown in Figure 2, the median SFR in the spec-$z$ sample is elevated by approximately $0.1~{}\mathrm{dex}$ (a factor of 1.3) as compared to the entire main sequence population ($K_{\mathrm{S}}\leq 23.5$) at the threshold stellar mass. We do not take this bias into account since it is well below the scatter in SFR and thus does not affect our estimate of the baryon conversion efficiency with the given statistics.

We calculate the mass accretion rates by using the halo mass accretion history derived by Behroozi et al. (2013a). With the baryon fraction of accreting matter fixed to the cosmic baryon fraction ($\Omega_{b}/\Omega_{m}=0.17$), the baryon accretion rate is found to be $\dot{M}_{\mathrm{baryon}}=43~{}M_{\odot}~{}\mathrm{yr^{-1}}$ for halos of $M_{\mathrm{halo}}=M_{\mathrm{min}}=10^{11.86}~{}M_{\odot}$. We note that the value of $\dot{M}_{\mathrm{baryon}}$ varies almost proportionally with halo mass. As a result, we find that the main sequence galaxies with $M_{\ast}\sim 10^{9.57}M_{\odot}$ at $z\sim 1.6$ convert $\epsilon_{\mathrm{baryon}}=\mathrm{SFR}/\dot{M}_{\mathrm{baryon}}\sim 35\%$ of baryons accreting onto halos into new stars. This estimate is consistent with the average conversion efficiency derived by Behroozi et al. (2013a). We further give a rough estimation on the scatter in $\epsilon_{\mathrm{baryon}}$ with an assumption that the scatter in SFR is caused by independent fluctuations in $\epsilon_{\mathrm{baryon}}$ and in $\dot{M}_{\mathrm{baryon}}$, which is now considered to be proportional to halo mass. Because the scatter in SFR is approximately $0.3~{}\mathrm{dex}$ and the scatter in halo mass is expected to be $0.2~{}\mathrm{dex}$ (see Appendix C) at fixed stellar mass ($=M_{\ast}^{\mathrm{lim}}$), the scatter in $\epsilon_{\mathrm{baryon}}$ is to be $\sim 0.2~{}\mathrm{dex}$ (a factor of 1.6) or less.

At $M_{\mathrm{h}}=M_{\mathrm{min}}$, the mass ratio $M_{\ast}/M_{\mathrm{h}}$ is likely to be proportional to $M_{\mathrm{h}}^{1.5}$, i.e., $M_{\ast}\propto M_{\mathrm{h}}^{2.5}$ (Martinez-Manso et al., 2015; Behroozi et al., 2013a). If the baryon conversion occurs at a constant rate for all halo masses, the stellar mass grows proportionally to the halo mass as the baryon accretion rate is expected to be almost proportional to halo mass, i.e., $\dot{M}_{\mathrm{baryon}}\propto M_{\mathrm{h}}$. Therefore, the stronger dependence of $M_{\ast}$ to $M_{\mathrm{h}}$ indicates that the integrated past baryon conversion depends on the halo mass. We recall that the slope of the star-forming main sequence ($M_{\ast}\textrm{--}\mathrm{SFR}$) is $\sim 0.7\textrm{--}0.8$ (e.g., Kashino et al., 2013). Given these observational facts, we find that the baryon conversion efficiency increases moderately with increasing stellar mass, as $\epsilon_{\mathrm{baryon}}\propto M_{\ast}^{0.3\textrm{--}0.4}$, and more strongly with halo mass as $\epsilon_{\mathrm{baryon}}\propto M_{\mathrm{h}}^{0.75\textrm{--}1.0}$. A similar dependence has also been found at a higher redshift (see Figure 13 of Harikane et al. 2016). We note that these relations pertain to the stellar mass between $10^{9.57}~{}M_{\odot}$ and $M_{\ast}\sim 10^{10.5}~{}M_{\odot}$, or halo mass from $M_{\mathrm{min}}=10^{11.86}~{}M_{\odot}$ to $M_{\mathrm{h}}\sim 10^{12.2}~{}M_{\odot}$. Since the slope of the SHMR decreases with increasing (both stellar and halo) mass (while eventually its slope becomes negative), the conversion efficiency has a peak value ($\epsilon_{\mathrm{baryon}}\sim 0.6$) that begins to decrease towards the highest masses (see Figure 11 of Behroozi et al. 2013a).

7.4. Satellite galaxies

The detection of the one-halo term in the correlation function enables us to estimate the fraction of galaxies that are satellites ($f_{\mathrm{sat}}$). This fraction is dependent on $M_{\mathrm{min}}$, $M_{1}$, and the halo mass function. For our sample, we find a satellite fraction $f_{\mathrm{sat}}=0.38_{-0.20}^{+0.14}$ from Equation (32). This is similar to the value of $f_{\mathrm{sat}}\approx 0.3$ for local galaxies (Zehavi et al., 2011). At higher redshifts, Wake et al. (2011) measure $f_{\mathrm{sat}}\approx 0.28$ for stellar mass limit samples ($>10^{10}M_{\odot}$) at $z\sim 1.5$. Martinez-Manso et al. (2015) also report a similar value ($f_{\mathrm{sat}}\sim 0.25$) at $z\sim 1.5$. Our result is consistent with these values as well. While our data do not constrain the redshift evolution of $f_{\mathrm{sat}}$ with the given statistical error, Coupon et al. (2012) suggested that the satellite fraction moderately increases with cosmic time since $z\sim 1$, which can be straightforwardly interpreted as a consequence of average growth of over-dense regions.

The ratio $M_{1}/M_{\mathrm{min}}$ determines the “shoulder” in $\left<N|M_{\mathrm{h}}\right>$ (Conroy et al., 2006), or the gap between halo masses at which halos acquire a central galaxy and host an additional satellite galaxy (refereed to as the “hosting gap” by Zehavi et al. 2011). Zehavi et al. (2011) found $M_{1}/M_{\mathrm{min}}\approx 17$ for local galaxies and that the ratio decreases with increasing threshold luminosity of a given sample. Here we find the ratio to be $M_{1}/M_{\mathrm{min}}=4.6_{-2.1}^{+4.5}$ for our FMOS sample, which is smaller than the local value and qualitatively consistent with the trend as reported by Conroy et al. (2006). Our result is in good agreement with previous findings at intermediate redshifts. For example, Wake et al. (2011) find $M_{1}/M_{\mathrm{min}}\approx 6$ for samples with stellar mass above $\sim 10^{10}~{}M_{\odot}$ at $1\lesssim z\lesssim 2$. Recently, Martinez-Manso et al. (2015) also found similar values $M_{1}/M_{\mathrm{min}}\approx 7$. Conroy et al. (2006) find a decrease in $M_{1}/M_{\mathrm{min}}$ with increasing redshift, which may reflect the increasing fraction of relatively low-mass halos that host satellites in addition to a central galaxy. The authors argue that pairs of galaxies within the same halo with mass close to $M_{\mathrm{min}}$ predominantly contributes to the one-halo term of the correlation function at higher redshifts.

7.5. Prediction for the future Subaru/PFS survey

Finally, we give predictions for the performance of the clustering measurements expected for the future Subaru galaxy survey using the Prime Focus Spectrograph (PFS; Tamura et al. 2016). This is a next-generation multi-object spectrograph that follows FMOS, having 6 times larger field-of-view (1.3 degree in diameter), many more fibers ($N_{\mathrm{fiber}}=2400$) than FMOS, and an unprecedented wavelength coverage ($0.38\textrm{--}1.26~{}\mathrm{\mu m}$). The preliminary PFS survey strategy is described in Takada et al. (2014), while the design has been modified in part to date. The PFS galaxy evolution survey aims to have a dedicated program to observe roughly half a million color-selected galaxies at $1<z<2$ to a limiting magnitude of $J_{\mathrm{AB}}\approx 23.3$ (or more deeper) over $\sim 15~{}\mathrm{deg^{2}}$. Galaxies will be selected from the Hyper Suprime-Cam Subaru Strategic Program (HSC-SSP) that provides deep and wide imaging with $grizY$ photometry.

Here, we aim to evaluate the statistical errors and how strongly the parameters can be constrained with a PFS-like sample, while assuming our best-fit values from the HOD modeling as fiducial. The signal-to-noise ratio of an observed correlation function is approximately proportional to the square root of the galaxy pair counts in bins of the separation. For an ideal survey with a contiguous survey volume and uniform sampling, the pair counts are approximately proportional to the survey volume and to the square of galaxy number density. If the survey field is separated into distinct smaller subregions, the number of galaxy pairs, especially with large spatial separations, increases more slowly than expected. The PFS survey will cover multiple distinct areas each having $\sim 6\textrm{--}7~{}\mathrm{deg}^{2}$. Such separate regions will not significantly affect the scales that we cover in this work ($\lesssim 20~{}\mathrm{cMpc}$). Therefore, we simply scale the statistical errors on the observed correlation function to match the expected survey volume and number density for the PFS survey.

For this purpose, we adopt the same redshift range as this work, i.e., $1.43\leq z\leq 1.74$, resulting in a comoving volume of $1.75\times 10^{7}(h^{-1}~{}\mathrm{cMpc})^{3}$ for a nominal survey area of $15~{}\mathrm{deg^{2}}$, which is 20 times as large as our FMOS survey. We assume a mass complete sample above $10^{10}~{}M_{\sun}$ with a uniform sampling rate of $75\%$. Based on the COSMOS catalog (Ilbert et al., 2013), the sample size should be $N\sim 6.5\times 10^{4}$ with the galaxy number density of $4\times 10^{-3}(h^{-1}~{}\mathrm{cMpc})^{-3}$, which is 7 times as high as our FMOS-spec-$z$ sample (see Table 2). As a very rough estimate, the errors on the correlation function will decrease by a factor of 30 for the full sample with the same binning in $r_{\mathrm{p}}$, while, if we divide the full sample equally into 10 subsamples, the errors become smaller by a factor of 3 as compared to our data. The values of $\log w_{\mathrm{p}}(r_{\mathrm{p}})$ are then adjusted by adding statistical noise corresponding to the scaled errors. Again, we do not intend here to make any predictions for the shape and/or absolute values of correlation function, but just to demonstrate the improvement in statistical accuracy.

For these two cases (the full sample with a 75% sampling rate and the 1/10 binned subsample), we show in Figure 21 the expected measurements and parameter constraints in comparison with our results. For simplicity, we assume the pair counting with the same ($r_{\mathrm{p}}$, $\pi$) grid and employ the same priors as our work with the FMOS data. As evident, the statistical uncertainties are remarkably improved for the hypothetic samples for a future survey, even if the samples are limited to a narrow range of redshift. For the full sample, it is obvious that the systematic effects that we have discussed in this paper, i.e., the impact of fiber allocation and/or inhomogeneous line-of-sight detection rate, will significantly influence the measurements, and thus the physical interpretations. In the right panel of Figure 21, it is shown that the constraints on $M_{\mathrm{min}}$ and $M_{\mathrm{1}}^{\prime}$ are significantly improved, while those on $\sigma^{2}_{\log M}$ are almost identical to our FMOS result. In fact, this parameter is essentially determined by the prior. This means that even very statistically-accurate clustering measurements cannot constrain all parameters strongly. Therefore, alternative methods are still necessary to fully understand the galaxy-halo relation. In conclusion, the future multi-object spectroscopic surveys (e.g., PFS, MOONS, DESI) assure significant improvement in the statistics with respect to clustering measurements, while systematic effects must be excluded more carefully and precise models are imperative to interpret such upcoming observational data.

8. Summary

We have investigated the clustering properties of star-forming galaxies at $1.43\leq z\leq 1.74$ using the data set from the FMOS-COSMOS survey. With 516 galaxies having an H$\alpha$ detection, we measured the projected two-point correlation function and investigated the properties of dark matter halos through the HOD modeling of galaxy clustering. Our main results are as follows:

The observed correlation function indicates a significant clustering at $0.04\lesssim r_{\mathrm{p}}/(h^{-1}~{}\mathrm{cMpc})\lesssim 10$. By modeling with a power-law function, we find a correlation length of $r_{0}=5.21^{+0.70}_{-0.67}~{}h^{-1}~{}\mathrm{cMpc}$, which is consistent with preceding studies using galaxy samples with stellar masses similar to our study. 2. 2.

We model the observed correlation function using an HOD model (Equations 29–30). The HOD parameters are effectively constrained with our current data and a significant one-halo term is confirmed with the transition scale of $r_{\mathrm{p}}\simeq 0.5~{}h^{-1}~{}\mathrm{cMpc}$, where the one-halo and two-halo terms are equivalent. 3. 3.

We derive an effective large-scale bias, $b_{\mathrm{eff}}=2.17^{+0.17}_{-0.24}$, and find that star-forming galaxies with $M_{\ast}\geq 10^{9.57}~{}M_{\odot}$ reside on average in halos with mass $M_{\mathrm{eff}}=4.62^{+1.14}_{-1.63}\times 10^{12}~{}h^{-1}M_{\odot}$, which is consistent with other measurements from HOD modeling. With predictions of the mass assembly histories, we find that these halos will have grown into group-scale halos ($\sim 2\times 10^{13}~{}h^{-1}~{}M_{\odot}$) at the present epoch. 4. 4.

Our work investigates the low-mass part of the stellar-to-halo mass relation at this redshift, with a new constraint, $\log M_{\ast}/M_{\mathrm{h}}=-2.3\pm 0.2$ at $M_{\mathrm{h}}=10^{11.86}~{}M_{\odot}$, while confirming the decline in the stellar-to-halo mass ratio at $M_{\mathrm{h}}<10^{12}~{}M_{\odot}$. However, there are discrepancies between SHMRs based on different observations and/or analyses. In particular, SHMRs based on HOD modeling, including our result, seem to have systematically lower values at lower halo masses than those based on abundance matching. 5. 5.

We find the efficiency of converting baryons into stars relative to the total amount of baryons accreting onto halos to be $\sim 35\%$ with a maximum scatter of a factor of $\sim 1.6$, and that the baryon conversion efficiency $\epsilon_{\mathrm{baryon}}$ depends on stellar and halo mass as $\epsilon\propto M_{\ast}^{0.3\textrm{--}0.4}$ and $\epsilon\propto M_{\mathrm{h}}^{0.75\textrm{--}1.0}$, respectively, around the threshold mass up to the peak mass of the SHMR ($\log M_{\ast}/M_{\odot}\approx 9.57\textrm{--}10.5$, $\log M_{\mathrm{h}}/M_{\odot}\approx 11.86\textrm{--}12.2$). 6. 6.

We find that the satellite fraction to be $f_{\mathrm{sat}}=0.38^{+0.14}_{-0.20}$, which is consistent with other measurements at both local and higher redshifts. In addition, the $M_{1}/M_{\mathrm{min}}$ ratio is found to be $\sim 5$, which is lower than as typically seen for low-$z$ galaxies ($\simeq 17$ at $z\lesssim 0.1$) while consistent with other studies at higher redshifts ($z>1$), which show $M_{1}/M_{\mathrm{min}}<10$. These results suggests that galaxy pairs in the same, relatively low-mass halos significantly contribute to the one-halo term of the correlation function.

In addition, we established correction schemes for fiber allocation and inhomogeneous detection due to sky contamination, and demonstrated the effectiveness of these methods even for small scales (i.e., the one-halo regime). Such techniques will be effective to measure the intrinsic galaxy clustering in the future spectroscopic surveys with the next-generation multi-object spectrographs such as Subaru/PFS. Future surveys will provide a sample of $\sim 5\times 10^{5}$ galaxies at $z>1$, and allow to measure galaxy clustering as a function of galactic properties at much elevated statistical accuracy. Therefore, the systematic effects must be excluded carefully and precise models are required to draw information maximally from such upcoming observational data.

This work is based on data collected at the Subaru telescope, which is operated by the National Astronomical Observatory of Japan. We greatly thank the staff of the Subaru telescope, especially K. Aoki, for supporting the observations. We greatly thank T. Ishiyama and the $\nu^{2}$GC team, and the Bolshoi simulation collaboration group for making their halo/subhalo catalogs public. We appreciate P. Behroozi for providing their data. J.D.S is supported by JSPS KAKENHI grand Number 26400221 and the World Premier International Research Center Initiative (WPI), MEXT, Japan. D.K. was supported through the Grant-in-Aid for JSPS Fellows (No. 26-3216).

Appendix A Correction for critical biases in the observed correlation function

A.1. Mock samples from the $\nu^{2}$GC simulation

We construct mock samples to evaluate our correction scheme for the effects of fiber allocation and sky contamination using the new numerical Galaxy Catalog ($\nu^{2}$GC) cosmological simulation (Ishiyama et al., 2015). We use the middle size simulation ($\nu^{2}$GC-M) that was conducted using $4096^{3}$ particles with a mass of $2.20\times 10^{8}~{}h^{-1}M_{\odot}$ in a comoving cube having a side length of $560~{}h^{-1}~{}\mathrm{cMpc}$. We employ a halo/subhalo catalog at a scale factor $a=0.384871$ ($z=1.598$), which is close to the median redshift of our spectroscopic sample. Halos and subhalos are identified using the Rockstar algorithm (Behroozi et al., 2013b), resolving subhalos with $M_{\mathrm{h}}\sim 10^{10}~{}h^{-1}M_{\odot}$. The lengths of our survey volume in the plane of the sky and along the line-of-sight are $\sim 60~{}h^{-1}~{}\mathrm{cMpc}$ and $\sim 400~{}h^{-1}~{}\mathrm{cMpc}$ over $1.43\leq z\leq 1.74$, respectively. Hence, we can divide the simulation box into 64 sub-boxes on the $x$-$y$ plane each having a volume of $70\times 70\times 560~{}(h^{-1}\mathrm{cMpc})^{3}$ that can enclose the survey volume. In doing so, we implicitly ignore the redshift evolution between this redshift range. Then, we convert the comoving coordinates ($x,y,z$) into angular coordinates and redshift, associating the middle point in a box to the center of the survey volume. We take the peculiar velocity along $z$-axis of individual (sub)halos into account to derive the true redshift (as opposed to the cosmological redshift).

For each subbox, we select halos and subhalos from the catalog to create a sample that has a number density similar to the FMOS-parent sample ($n=3.07\times 10^{-3}~{}h^{3}\mathrm{cMpc}^{-3}$) and a correlation function whose amplitude is also similar to the observed one (see Figure 24). We extract halos having a maximum circular velocity $V_{\mathrm{max}}$ greater than $170~{}\mathrm{km~{}s^{-1}}$ and subhalos having $V_{\mathrm{max}}>160~{}\mathrm{km~{}s^{-1}}$. It is known that the highest $V_{\mathrm{max}}$ that a halo can attain over its past history (often called $V_{\mathrm{peak}}$) is a better proxy of stellar mass, rather than $V_{\mathrm{max}}$ at the epoch of interest, especially in order to better reproduce galaxy clustering (Reddick et al., 2013). This is reflective of the fact that subhalos lose their mass and $V_{\mathrm{max}}$ when they accrete onto their parent halos. However, such information is not yet available for this simulation. To roughly account for the mass loss of subhalos, we here use a lower threshold of $V_{\mathrm{max}}$ for subhalos than for parent halos. However, the precise selection does not matter because we are just interested in the relative change due to the observational biases, but not in any absolute quantities predicted by the simulation.

To ensure that the redshift distribution of the mock samples matches that of the FMOS-parent sample, we assign photometric redshifts with additional random fluctuations of $\mathrm{rms}(\Delta z)=0.062$ to the true redshift of each halo. Then we extract objects from the same range of the photometric redshift as we applied for the real data ($1.46\leq z_{\mathrm{phot}}\leq 1.72$). From them, we randomly select $\sim 2300$ objects to match the number of the FMOS-parent sample, and refer these as the Mock-parent samples. From these samples, we construct two types of mock samples that take into account the inhomogeneous detection along the line-of-sight (mainly due to the atmospheric lines) and fiber allocation effects.

Then we randomly select the same number of objects as present in our FMOS-spec-$z$ sample (516) from a subset of the Mock-parent sample restricted to having a true redshift $1.43\leq z\leq 1.74$. During this selection, we account for the weight as a function of redshift, as shown in Figure 7, to reflect the non-uniform detectability of the H$\alpha$ emission line along the line of sight. These Mock-zweight samples are used in the subsequent section.

Next we prepare mocks to assess the impact of fiber allocation algorithm. For each Mock-parent sample, we mimic the selection function induced by the FMOS fiber allocation software to construct the Mock-fiber-target samples. We also account for fibers which broke down, some of these were regularly out-of-order and the others depending on the observing runs. These samples have approximately the same number of galaxies ($\sim 1200$) as the FMOS-fiber-target sample, representing the observed (fiber-allocated) galaxies. Then we create the Mock-spec-$z$ samples by randomly extracting 516 objects with a true redshift between $1.43\leq z\leq 1.74$ from each Mock-fiber-target catalog. These final samples correspond to the FMOS-spec-$z$ sample that is used for clustering measurement. We note that the fiber allocation is not considered for the Mock-zweight samples, and the inhomogeneous detection rate is not taken into account for the Mock-fiber-target/spec-$z$ samples to examine each of these effects individually.

A.2. Inhomogeneous line-of-sight detectability

The inhomogeneous detection of the H$\alpha$ emission line along the redshift direction due to the OH lines/mask and the instrumental characteristics can induce an artificial clustering signal along the direction parallel to the line of sight. As a result, the amplitude of the correlation function is expected to be artificially enhanced if corrections are not applied. In our analysis, we used a random sample where the redshifts were sampled from the intrinsic distribution of the FMOS-parent sample, but with a probability weighted by the non-uniform detectability as a function of redshift (Figure 7).

We demonstrate the effects and the reliability of our method. We calculate the correlation functions of a series of Mock-zweight samples, in which the non-uniform detectability is taken into account by selecting 516 objects with the weighted probability (Appendix A.1). To assess the effect on our calculation of the correlation function using Equation (2), we inspect differences in the clustering using random samples with and without the non-uniform detection rate corrections. In Figure 22, the average correlation function of 64 mock samples is shown corresponding to the use of these random catalogs, and compared to the intrinsic correlation function of the sample. It can be seen that the use of the modified random samples accurately recovers the intrinsic one (filled red circles). In contrast, the values of $w_{\mathrm{p}}(r_{\mathrm{p}})$ are slightly, but systematically enhanced over the range of $r_{\mathrm{p}}>0.1~{}h^{-1}\mathrm{Mpc}$ if we do not consider the non-uniform weight in the random sample (open blue circles). This enhancement of the clustering amplitude results in an overestimate of the correlation length by $\Delta r_{0}\sim 0.2\textrm{--}0.3~{}h^{-1}~{}\mathrm{cMpc}$. Although this systematic error is well below the uncertainty on the measurement from our current data, it could be critical in future studies based on large samples, which provide much higher statistical accuracy than our study (see Section 7.5). This exercise evidently indicates that the effects of non-uniform detectability along the redshift direction can be mitigated by applying an appropriate weight function for the random sample.

A.3. Fiber allocation

The selection of targets is dependent on the software that allocates fibers to objects (Akiyama et al., 2008). As a result, there can be artificial biases in the on-sky distribution of objects since not all galaxies in the input catalog are observed. After allocating a pair of fibers to one galaxy, the opportunity for its neighboring galaxies to be observed at the same time decreases due to the lack of fibers and to avoid fiber entanglement. In contrast, galaxy pairs having separations of several times the fiber separation are easy to simultaneously assign as targets. This results in a bias in the pair counts, hence the observed correlation function. In addition, the sampling rate varies across the survey area because of the different number of exposures among the four footprints and the existence of the small areas where the footprints overlap. Moreover, some broken fibers result in holes, where the sampling rate is reduced. In order to correct for these biases, we apply a weight for each galaxy pair as a function of their angular separation (see e.g., de la Torre et al., 2011; Durkalec et al., 2015a)). The weight is defined by Equation (12) using the ratio of the angular correlation function of the input catalog and that of the sample of galaxies for which fibers were allocated.

In Figure 23, we demonstrate the effects of fiber allocation. The angular correlation functions $\omega(\theta)$ of the real data, FMOS-parent and FMOS-fiber-target samples, are shown in panel (a). It is evident that the angular clustering amplitude of the FMOS-fiber-target sample is suppressed compared to the FMOS-parent sample at scales $\theta\lesssim 100~{}\mathrm{arcsec}$. This angular scale is similar to the fiber separation, corresponding to $r_{\mathrm{p}}\lesssim 2~{}h^{-1}~{}\mathrm{cMpc}$ at $z\sim 1.6$. We also show the angular correlation functions for 64 sets of Mock-parent and Mock-fiber-target samples. Although the scatter is large, the suppression at small scales is evident in their average values, in agreement with the real data. This decrease in the angular clustering amplitude is expected to results from the fiber allocation. Figure 23b shows the ratios of $1+\omega(\theta)$ between the parent and fiber-target samples for both the data (red circles) and the mocks (light blue dots). For our analyses, we use the average ratios calculated from the mock samples (thick solid line) as the weight function (Equation 12). The weight function is $>1$ at angular scales of $\theta\lesssim 100$ arcsec and reaches $\sim 1.2$ at $\theta<20$ arcsec, while it is slightly smaller than unity at scales at several hundreds arcsec. On large scale, it reflects the ease of observing pairs with a separation several times larger than the fiber separation simultaneously.

We examine the effectiveness of our correction scheme. In Figure 24, we compare the average correlation functions $w_{\mathrm{p}}(r_{\mathrm{p}})$ of 64 Mock-spec-$z$ samples with and without the correction to the intrinsic correlation function. The average amplitude of the Mock-spec-$z$ samples without the correction is reduced at scales $r_{\mathrm{p}}\lesssim 2~{}h^{-1}~{}\mathrm{cMpc}$, while it is slightly enhanced at $r_{\mathrm{p}}\gtrsim 3~{}h^{-1}~{}\mathrm{cMpc}$, as clearly seen in the lower panel. In contrast, the application of the correction scheme recovers the true $w_{\mathrm{p}}(r_{\mathrm{p}})$ over the entire scale of interest in a relatively unbiased manner. These biases are not negligible even for studies using a relatively small sample, as in our study, and necessarily will has a more significant impact for studies with larger samples. We conclude that the proposed correction scheme based on the angular correlation functions works appropriately in our analysis (and should work for future multi-object surveys) even for small scales down to $r_{\mathrm{p}}\sim 0.1~{}h^{-1}~{}\mathrm{cMpc}$.

Appendix B The effects of sample selection

We assess the effect of the stellar mass completeness on the constraints of the HOD model. For this purpose, we construct mock samples using the Bolshoi simulation (Klypin et al., 2011) to take the advantage of its better mass resolution and the availability of merger histories of halos, instead of the $\nu^{2}$GC simulation used above. The simulation consists of $2048^{3}$ particles with a mass of $1.35\times 10^{8}~{}h^{-1}M_{\odot}$ in a comoving cube having a side of $250~{}h^{-1}~{}\mathrm{cMpc}$. The volume of the simulation box is 19 times as large as our survey volume, while the side length is shorter than the radial extend of the survey ($\sim 380~{}h^{-1}~{}\mathrm{cMpc}$). We utilize the public catalog at $a=0.38435$ ($z=1.602$), in which halos are identified down to a mass of $10^{10}~{}h^{-1}M_{\odot}$ based on the Rockstar halo finding algorithm (Behroozi et al., 2013b). For the following exercise, we use the same cosmology as the Bolshoi simulation $(\Omega_{m},\Omega_{\Lambda})=(0.27,0.73)$.

For abundance matching, we use the peak maximum circular velocity ($V_{\mathrm{peak}}$) provided in the public catalog as the indicator of the stellar mass of a galaxy hosted by a (sub)halo. Assuming a tight correlation between $V_{\mathrm{peak}}$ and stellar mass with no scatter, we chose the lower limit $V_{\mathrm{peak}}^{\mathrm{lim}}$ by matching the (sub)halo number density $n\,(>V_{\mathrm{peak}})$ to the number density of the $M_{\ast}$-selected galaxy sample (Table 2). Note that the number density is recomputed, as $n=8.47\times 10^{-3}(h^{-1}\mathrm{cMpc})^{-3}$ based on the above cosmology. In total, 132408 (sub)halos with $V_{\mathrm{peak}}>V_{\mathrm{peak}}^{\mathrm{lim}}\approx 165~{}\mathrm{km~{}s^{-1}}$ are selected. For each (sub)halo, a stellar mass is assigned, while keeping the relation $n\,(>V_{\mathrm{peak}})=n\,(>M_{\ast})$, in order to reproduce the stellar mass function of the $M_{\ast}$-selected sample (see Figure 3). We denote this mock catalog as Mock-$V_{\mathrm{peak}}$-limited sample. From this sample, we construct the Mock-$M_{\ast}$-incomplete sample, in which the stellar mass incompleteness is taken into account, by randomly select (sub)halos with a probability as a function of the assigned stellar masses, such that the resulting stellar mass function equals that of the FMOS-parent sample (Figure 3). In total, 43777 (sub)halos are selected.

We measure the projected correlation function of these selected halos. Figure 25 compares the correlation functions for the Mock-$V_{\mathrm{peak}}$-limited (red triangles), Mock-$M_{\ast}$-incomplete (blue squares), and the FMOS-spec-$z$ (black circles) samples. We correct the correlation functions of the mock samples for the effect of the finite size of the simulation box (see van den Bosch et al., 2013, for details). We also note that the correlation function for the real data is multiplied by $1/(1-f_{\mathrm{fake}})^{2}$ where $f_{\mathrm{fake}}=0.099$, which is the expected value from the HOD fitting (Table 4). It can be seen that the clustering amplitude of the Mock-$M_{\ast}$-incomplete sample (blue squares) is slightly enhanced relative to the Mock-$V_{\mathrm{peak}}$-limited sample (red triangles). This is naturally expected from the fact that the $M_{\ast}$-incomplete sample is on average biased towards massive halos, which are more strongly clustered. We fit the HOD model (Equations 29 and 30) to the correlation functions of these two mock samples. Here we scale the covariance matrix of the mock measurements by a factor of 19, which is the ratio of the volumes of the simulation box and the survey, to match the level of the statistics to the real data. The parameter constraints are shown in Figure 26 separately for the $V_{\mathrm{peak}}$-limited and $M_{\ast}$-incompleteness samples in comparison to the results for the real FMOS sample. We find that all parameters are consistent within their 1 $\sigma$ confidence level, although the peak positions of the posterior distributions are slightly different among data and mocks. In particular, the constraints are almost identical for the two mock samples. Theses results indicate that the stellar mass incompleteness in our FMOS sample should not have a significant impact on our conclusions, given the current level of statistical errors.

Appendix C Limitation on $\sigma_{\log M}$

In this section, we present the prior that we assign to the HOD parameter, $\sigma_{\log M}$ (see Section 6.2). We use the SHMR derived by Behroozi et al. (2013a), who parametrized its evolution and intrinsic scatter over $0<z<8$ and constrained it using a variety of observational constraints. Although our constraint on the stellar-to-halo mass ratio deviates from the Behroozi et al. (2013a) SHMR (see Section 7.2), this discrepancy is not significant in the following calculations because only the scatter and the slope of the SHMR matter for the prior that we use for $\sigma_{\log M}$.

The scatter in stellar masses ($\log M_{\ast}$) of galaxies that inhabit halos of a given mass is constrained to be $\sigma_{\log M_{\ast}}^{\mathrm{int}}=0.23\pm 0.04$ at $z=1.588$. In addition, we incorporate another scatter ($\sigma^{\mathrm{sample}}_{\log M_{\ast}}$) which accounts for sample selection. We select galaxies based on the best estimate of the stellar mass from the SED, which should differ from the true value due the uncertainties in the photometric redshift, flux measurement, and the population synthesis model. Therefore, a threshold in the estimated stellar mass does not correspond to a threshold in the true stellar mass, and one can expect low stellar mass objects to scatter into our sample. The width of this scatter will correspond to the typical error in the stellar mass estimation. We compute for each galaxy the probability that its true stellar mass satisfies $M_{\ast}\geq M_{\ast}^{\mathrm{lim}}$, and derive the probability-weighted stellar mass distribution of our sample such that it has a smoothed lower limit at $M_{\ast}=M_{\ast}^{\mathrm{lim}}\equiv 10^{9.57}~{}M_{\odot}$. We find $\sigma^{\mathrm{sample}}_{\log M_{\ast}}=0.09$ by deconvolving the distribution into a sharp-cutoff function and a gaussian function with a standard deviation $\sigma^{\mathrm{sample}}_{\log M_{\ast}}$. The total scatter in $\log M_{\ast}$ of galaxies within halos of the threshold mass is thus given as

[TABLE]

We then calculate the probability of finding a galaxy of $M_{\ast}=M_{\ast}^{\mathrm{lim}}$ in a halo as a function of halo mass:

[TABLE]

where $M_{\ast}(M_{\mathrm{h}})$ is the SHMR parametrized by Behroozi et al. (2013a). The expected mean and mean square of halo mass to have a galaxy of $M_{\ast}=M_{\ast}^{\mathrm{lim}}$ are given by

[TABLE]

where $n(M_{\mathrm{h}})$ is a halo mass function at a given redshift, and $m=1(2)$ corresponds to the mean (mean square). We then find the deviation in halo mass at our threshold stellar mass:

[TABLE]

The HOD parameter $\sigma_{\log M}$ is then given by $\sigma_{\log M}=\sqrt{2}\times\sqrt{\mathrm{Var}(\log M_{\mathrm{h}})}=0.24\pm 0.3$. This is the prior information we adopt for the parameter in our HOD modeling. We have checked that the inclusion of the redshift evolution of the Behroozi et al. (2013a) SHMR over $1.43\leq z\leq 1.74$ does not change these values. The derived value of $\sigma_{\mathrm{\log M}}$ is consistent with an estimate in previous studies (e.g., More et al., 2011), albeit at local redshifts.