Median statistics estimates of Hubble and Newton's Constant

TL;DR

This paper demonstrates the robustness of median statistics through toy experiments and applies it to estimate the Hubble Constant and Newton's Gravitational Constant, revealing significant measurement discrepancies.

Contribution

It introduces median statistics as a robust method for estimating fundamental constants, updating previous analyses with a larger dataset and providing new median estimates.

Findings

Median estimate of Hubble Constant: 72.5 km/sec/Mpc with 95% confidence interval.

Median estimate of Newton's G: 6.674702 × 10^{-11} N m^2 kg^{-2} with 68% confidence.

Median statistics show robustness against assumptions and measurement variations.

Abstract

Robustness of any statistics depends upon the number of assumptions it makes about the measured data. We point out the advantages of median statistics using toy numerical experiments and demonstrate its robustness, when the number of assumptions we can make about the data are limited. We then apply the median statistics technique to obtain estimates of two constants of nature, Hubble Constant ($H_0$) and Newton's Gravitational Constant($G$), both of which show significant differences between different measurements. For $H_0$, we update the analysis done by Chen and Ratra (2011) and Gott et al. (2001) using $576$ measurements. We find after grouping the different results according to their primary type of measurement, the median estimates are given by $H_0=72.5^{+2.5}_{-8}$ km/sec/Mpc with errors corresponding to 95% c.l. (2$\sigma$) and $G=6.674702^{+0.0014}_{-0.0009} \times 10^{-11} \mathrm{N m^{2}kg^{-2}}$ corresponding to 68% c.l. (1$\sigma$).

Tables

Table 1. Table 1: Time read by nine watches along with the uncertainties of each watch in minutes. We also provide a unique “Build ID” for each watch in the third column. These uncertainties are chosen in an ad-hoc fashion with one outlier deliberately introduced.

Time	Uncertainty (in minutes)	Build ID
11:50 AM	$\pm$ 25	2
12:49 PM	$\pm$ 55	2
12:52 PM	$\pm$ 4	3
12:59 PM	$\pm$ 36	3
01:02 PM	$\pm$ 7	1
01:07 PM	$\pm$ 5	2
01:10 PM	$\pm$ 70	1
01:27 PM	$\pm$ 240	3
02:35 PM	$\pm$ 3	1

Table 2. Table 2: Robustness of Median statistics: Here, we generate random samples from three separate distributions with different sizes and means. Then, we compute the empirical mean and median of each of the samples and tabulate the result. We notice that our estimate based on mean statistics is sensitive to outliers, whereas our median is approximately equal to the true mean.

Distribution	Sample size	Parameters	Calculated Mean	Median	95% c.l. for Median
Gaussian	$1000$	$-10$	$-9.997$	$-9.993$	$[-9.996,-9.991]$
	$2000$	$10$	$10.020$	$10.029$	$[10.027,10.031]$
	$3000$	$0$	$0.0107$	$0.0104$	$[0.0092,0.0106]$
Cauchy	$1000$	$-10$	$-10.201$	$-10.1384$	$[-10.148,-10.1381]$
	$2000$	$10$	$8.513$	$10.044$	$[10.043,10.045]$
	$3000$	$0$	$-2.234$	$-0.007$	$[-0.008,-0.006]$
Pareto with $\alpha \le 1,x_m=1$	$1000$	$1/2$	$15549.92$	$3.663$	$[3.660,3.665]$
	$2000$	$1/3$	$607468177$	$8.465$	$[8.389,8.494]$
	$3000$	$1/4$	$8.663e+12$	$16.079$	$[16.047,16.096]$

Table 3. Table 3: Here, we group all the watches based on their “Build ID” and calculate the group median. We then calculate the median of medians and show that this value is very close to the true value. We note that this simulated data has been designed (by choice) so that the median of medians results in a better estimate of the true value.

Build ID	Group	Group Median	Median of Medians
1	01:02 PM	01:10 PM	12:59 PM
	01:10 PM
	02:35 PM
2	11:50 AM	12:49 PM
	12:49 PM
	01:07 PM
3	12:52 PM	12:59 PM
	12:59 PM
	01:27 PM

Table 4. Table 4: H 0 subscript 𝐻 0 H_{0} :Primary type grouping. Here, we group the updated list of H 0 subscript 𝐻 0 H_{0} measurements on the basis of “Primary type” and perform median statistics analysis. “Type” is the class name. “Number” is the strength of the group. “Median” is the median of that group. “95% Confidence limits” is the lower and upper confidence limits. The categories are the same as in R11.

Type	Number	Median	95% Confidence limits
Global Summary	118	70.0	69.0	71.0
Type Ia Supernovae	97	64.0	60.0	66.0
Other	85	68.0	60.0	71.0
Lens	84	64.5	62.0	69.0
Sunyaev-Zeldovich	46	60.5	57.0	66.0
Baryonic Tully-Fisher	23	60.0	56.0	71.0
Infrared Tully-Fisher	19	82.0	65.0	90.0
Fluctuations	18	75.0	71.0	81.0
Tully-Fisher	18	72.5	68.0	74.0
CMB fit	16	69.5	58.0	71.0
Globular Cluster Luminosity functions	14	76.5	65.0	80.0
$D_n-\sigma$/Fund plane	10	75.0	67.0	78.0
Inverse Tully-Fisher	9	74.0	69.0	77.0
Type II Supernovae	8	59.5	52.0	76.0
Planetary Nebula Luminosity Functions	6	85.0	77.0	87.0
Novae	4	77.0	-	-
Red Giants	1	74.0	-	-

Table 5. Table 7: H 0 subscript 𝐻 0 H_{0} :Complement set of Secondary type grouping. Here, we tabulate a set of measurements which do not belong to a specific “Secondary Type” and perform median statistics analysis. “Type” is the class name. “Number” is the strength of the group. “Median” is the median of that group. ‘ ‘95% Confidence limits” is the lower and upper confidence limits.

Type	Number	Median	95% Confidence Limits
Theory with assumed Omega	571	69.0	68.9	69.0
results presented at Irvine Conf	570	69.0	69.0	69.0
deVaucouleurs or van den Bergh	554	68.0	68.0	68.0
Key Project or Key Project team Member	513	67.0	67.0	67.0
Sandage and/or Tammann	504	70.0	70.0	70.0
Cosmology dependent	492	69.0	69.0	69.0
No second type	246	67.0	67.0	67.0

Table 6. Table 8: G 𝐺 G :Device grouping- We have grouped all measurements by “Device” and computed the group-wise median and 95% confidence interval. “Number” denotes the strength of each group. Please note that Median and 95% confidence limits have a multiplication factor of × 10 − 11 absent superscript 10 11 \times 10^{-11} and units are in Nm 2 ​ kg − 2 superscript Nm 2 superscript kg 2 \mathrm{Nm^{2}kg^{-2}}

Device	Median x${10}^{-11}$	Number	95% Confidence interval x${10}^{-11}$
Torsion Balance	6.674255	17	6.67346	6.67553
Two Pendulums	6.67328	2	-	-
Atom Interferometer	6.67191	1	-	-
Beam Balance	6.67425	1	-	-

Table 7. Table 12: This table summarizes the outcome of this paper. Here, groups correspond to the “group by” Primary and secondary type in case of H 0 subscript 𝐻 0 H_{0} and, Device and Mode in case of G 𝐺 G . Since, each constant has one “group by”, there are total of two cases for each constant. For H 0 subscript 𝐻 0 H_{0} , the first two values in “Table Wise Median” column correspond to Primary type , Secondary type. In case of G 𝐺 G , the two values correspond to Mode , Device . We are specifying 68% c.l. only for Mode “group” as the strength of the group is too less to compute 95% c.l. For the remaining case, c.l. corresponds to 95% c.l. In the Median and 95% c.l , we take global median and 95% c.l. using the two tables we computed for each Fundamental Constant. Please note that for G 𝐺 G , the values in all the rows corresponding to it have a multiplication factor of × 10 − 11 absent superscript 10 11 \times 10^{-11} .

Constant	Group Median	c.l.		Global Median	95% c.l.
$H_0$	72.5	64.5	75.0	$69.75$ km/s/Mpc	64.5	74.0
	68	52.5	72.5
$G$	6.674702	6.67328	6.675565	6.674252 ${\mathrm{Nm}}^2{\mathrm{kg}}^{-2}$	6.67191	6.675565
	6.673765	6.67191	6.67425

Equations

$$P_i=\frac{1}{2^N}\left(\genfrac{}{}{0ex}{}{n}{i}\right)$$

Full text

1 Introduction

Rapid advances in observational cosmology due to avalanche of new data have led to the era of “precision cosmology” with many of the key cosmological parameters determined to about 1% precision. The current best cosmological constraints come from Cosmic Microwave Background anisotropy measurements from the Planck satellite Planck . These constraints are expected to be measured to even better precision with ongoing CMB experiments and a variety of optical photometric and spectroscopic surveys such as eBOSS, DES, Euclid, HSC, KiDS etc.

However, despite this, there is still no consensus for over half a century on the measurements of first ever cosmological parameter introduced in literature, viz. the Hubble constant ($H_{0}$), which measures the expansion rate of the universe. Until the 1990s, $H_{0}$ ranged from 50 to 100 km/sec Mpc. (See for example the contradictory points of views as of 1996 on measurements of $H_{0}$ between Tammann Tammann and Van Den Bergh vandenberg .) The tension between different measurements of $H_{0}$ continues to persist in 2016 in the era of precision cosmology Verde ; Silk . Currently there is a 3.3$\sigma$ tension between the latest Planck constraint PlanckH0 ($H_{0}=66.93\pm 0.62$ km/s/Mpc) and local measurements from Hubble Space Telescope ($H_{0}=73.24\pm 1.74$ km/s/Mpc) based on Cepheid variables Riess .

Similar conflicting measurements have also been reported for measurements of Newton’s Gravitational Constant $G$. This constant appears in Newton’s law of universal gravitation and Einstein’s field equations in General Relativity. $G$ is notoriously hard to measure and the statistical significance of the tensions between different measurements is about $10\sigma$ Quinn ; Mohr . There have also been claims of periodicities in measurements of $G$ Anderson15a , which however have been disputed Pitkin ; Desai .

Here, we use median statistics technique (introduced by Gott et al Gott (hereafter G01) to combine results from all the latest measurements of $H_{0}$ and $G$ to calculate the central values and associated uncertainties. We note that this technique was previously applied to about 553 measurements of $H_{0}$ as of 2012 by Chen and Ratra Chen (hereafter R11). Median statistics has also been used to estimate values of other cosmological parameters Crandall , in SN1a data analysis Barreira , mean matter density Chen03 , and cosmological parameter estimation from SZ and X-ray observations Sereno .

The outline of this paper is as follows. In Sec. 2, we introduce the concept of median statistics, including calculation of confidence intervals and and demonstrate its advantages over mean statistics using toy numerical experiments. We then apply the median statistics technique to measurements of $H_{0}$ in Sec. 3 and $G$ in Sec. 4. We conclude in Sec. 5.

2 Median statistics

The central limit theorem NR says that under certain conditions, the arithmetic mean of a sufficiently large number of iterations of independent random variables, each with a well-defined (finite) expected value and finite variance, will approximately be normally distributed, regardless of the underlying parent distribution. If we consider each measurement ($X$) to be independent, we can expect the distribution of $X$ to be Gaussian.

One of the interesting features of a Gaussian distribution is that it’s mean and median are exactly the same. This opens up the possibility of using the median of a dataset for measurements when the mean is not adequate. Median statistics estimate can be obtained by minimizing sum of absolute deviation or L1 norm NR . In the forthcoming sections we build upon this idea further using toy numerical simulations and finally use the “Median Statistics” technique to obtain best-fit values of $H_{0}$ and $G$. For an accurate treatment of the median statistics on a given dataset, one should make the following assumptions:

All the measurements are independent

This implies that the rank of the next observation is random and is equally likely to happen between any of the previous measurements.

There is no overall systematic error

If there exists an overall error, which affects all the observations with the same magnitude as the statistical error, median statistics will lead to the wrong result (as we shall show at the end of subsection 2.2). An important point to realize here is that even though there cannot be an error affecting the whole data, it is possible to have different errors and sources of errors affecting mutually exclusive groups of data. We elaborate on this further in subsection 2.2. In case of the Gaussian distribution analogy discussed before, the bell curve would be shifted and hence, wouldn’t be equal to the TRUE VALUE. Median statistics is immune to outliers, which otherwise affect our mean. In Gott et al Gott , they have shown for a Cauchy distribution how the robust median and the 95% c.l. are well-behaved unlike the mean and variance, which are plagued by outliers. They have calculated the empirical mean and c.l., which they compared with the median and it’s c.l. They note that c.l. range for mean is lot more than that for median. This shows the robustness of Median statistics. The mean of a dataset can be easily biased by adding/removing few extreme values, however the median remains insensitive.

The median is agnostic to measurement errors

We shall elaborate upon the “watch” analogy (originally introduced by Zeldovich) described in G01. Let us consider nine watches for which the measurement of time from each watch is independent and its associated error is known $apriori$. The times shown by each watch and associated uncertainties can be found in Table 1 with the true time equal to 1:00 PM.

We find that the median time, 1:02 PM, turns out to be something close to the true value, viz. 1:00 PM. We need to employ different ways to calculate the uncertainty in median. Sufficient care has been taken to ensure that the time the watches read and the uncertainties they have are consistent. On an average, we expect watches to be inaccurate to within few minutes. So, uncertainties are chosen in such a way that the TRUE time lies in the interval around the watch time with uncertainty about the same as the half width.

2.1 Evaluating confidence levels

We briefly summarize the procedure for calculating the median confidence levels (c.l.), following G01 and R11, wherein more details can be found. The median of an array is defined as the $50\%$ percentile. Given $x\in$ array, then, there would be $i$ elements which are $\leq x$ and $(n-i)$ elements $>x$ for some $i$, where $n$ is the length of the array.

Since we know that each given measurement is random and its rank completely arbitrary, we can show that there are ${n\choose i}$ ways to choose $i$ from $n$ measurements. Furthermore, we assume that each measurement has equal probability of being $\leq x$ or $>x$, or has a probability equal to $\frac{1}{2}$. Let $P_{i}$ be the probability that $i$ element to be the TRUE MEDIAN. $P_{i}$ is then given by:

[TABLE]

Using this formula, we compute $P_{i}$ at every $i$ and define $C_{j}$ as sum of all $P_{i}$’s for $i$ ranging from $j:(N-j)$. To evaluate the 95% confidence limit indices about the median, we choose the minimum value of $r$ for which $C_{r}\geq 0.95$ where $C_{r}\in(C_{j}:C_{N-j})$ for all values of $j$ between 1 and $\frac{N}{2}$.

Using the above procedure, we find that the $95\%$ confidence limits ranges from 12:49 PM to 01:10 PM. Our error is directly proportional to $1/\sqrt{N}$ Chen . This is the same as what we would expect from Gaussian mean statistics. This implies that our median statistics is able to obtain this value without positing anything about the error distribution.

Median Confidence limits

We understand from the methodology discussed in the previous point, that the median confidence limit is a function of the sample size, $N$ and does not depend, in any way, on the sample distribution. In the limiting case, when $N=\lim_{n\to\infty}$, we find that the $100\%$ confidence limits index ($r$ in the previous section) is that of the median only. This agrees with our intuition that larger the size of our dataset, smaller is the range of $95\%$ confidence intervals. To further drive home this point, see Table 2.

An alternative to our approach of Median statistics would be Bayesian analysis Parkinson ; Trotta (and references therein), wherein one uses Bayes’ rule to determine the conditional probability of the correct observation (measurement) given the other observations in hand. Although detailed comparison of the two methods is beyond the scope of this paper and depends on the data been analyzed, usually Bayesian parameter estimation is expected to be more accurate compared to median estimates. In Bayesian analysis, one needs to posit a prior (based on previous measurements or expectations from theoretical models) used for the parameters or the model. Bayesian analysis is also computationally expensive compared to the calculation of the median. The Bayesian approach was first used by Press Press for $H_{0}$ values to marginalize over the bad measurements. Most recently, $H_{0}$ was determined from Cepheid data by using Bayesian hyper-parameters Cardona .

2.2 Effects of systematic errors

One of the strongest assumptions we make about the dataset is that it contains no overall egregious errors. This assumption does not preclude us from saying that mutually exclusive groups may have similar systematic errors affecting them. If one were to extend the nine watches analogy to demonstrate this effect, one would have to provide extra information about the watches. This extra information is usually some way to classify the types of watches for example, eg. Model number, Batch code, Manufacturer name, etc. The only constraint the extra information has to satisfy is to be able to classify each observation uniquely into mutually exclusive groups. The result can be summarized in Table 3.

Given any additional information about the nine watches, our estimate of the true median is more accurate than evaluating the median of the whole data, which gives a value of 1:02 PM. To summarize, we have taken into account the systematic error which affects the mutually exclusive groups encompassing our data.

We should point out one caveat related to the median estimate if the data contains systematic errors with a simple example. If we add an offset of 30 minutes to the time shown by each watch in our dataset, our median of medians changes to 01:29 PM. Since the true value is close to 01:00 PM we can essentially get any value for the median of medians by adding/subtracting an arbitrary value to all our watches. A way to counter this is to hypothesize that there are NO overall systematic errors affecting our entire dataset.

We now apply the median statistics method to measurements of Hubble’s constant $(H_{0}$) and Newton’s Gravitational Constant $(G)$.

3 $H_{0}$ : Hubble’s constant

G01 introduced the notion of median statistics and carried out this analysis for $331$ published estimates of $H_{0}$. This was updated by R11, using 553 measurements of $H_{0}$ compiled by J. Huchra. We replicate the analysis of R11 with updated measurements and present median statistics estimates using $576$ values of $H_{0}$ (updated as of Sept. 2016) . We do not include any errors in our analysis. The full list of all $H_{0}$ measurements is uploaded on google docs.

The median value of all $H_{0}$ measurements along with 95% c.l. without any grouping is given by $H_{0}=69.75\pm 5.25$ km/sec/Mpc. The extra information in the context of $H_{0}$ becomes the methodology employed to estimate $H_{0}$.

R11 classified all measurements of $H_{0}$ into $18$ primary types and $5$ secondary types. Using this well-maintained list, one can calculate the median of each subgroup and also the median of all these medians. Furthermore, one can even calculate the 95% c.l. as explained in subsection 2.1. The primary types grouping has been done based on the procedure employed to measure $H_{0}$ and is therefore a good way to classify measurements. The secondary types are more concerned with non-procedural factors involved in the measurement, namely, determinations by one group (Sandage), indirect measurements based on the underlying cosmological models, etc.

In Table 6, we have grouped all measurements according to primary type. And, in Table 6, we do the same but group on the basis of secondary type. Similar to R11, we also calculate the median values after excluding all measurements of primary and secondary type. These are shown in  Table 6 and  Table 7 respectively. The sub-group median values of both the primary and secondary types are shown in  Table 12. The median estimate after grouping according to primary type is given by $H_{0}=72.5^{+2.5}_{-8}$ km/sec/Mpc. After grouping according to secondary type, we get $H_{0}=68^{+4.5}_{-15.5}$ km/sec/Mpc.

4 $G$ : Newton’s Gravitational Constant

We now apply the median statistics technique to do similar analysis of $G$. For this purpose, we use the tabulated measurements of $G$ from Schlamminger et al Schlamminger , who have a compiled a list of all $G$ measurements since 1980. A review of all previous measurements (starting from the first one by Cavendish in 1798) and associated controversies are reviewed in Refs. Quinn ; Mohr ; Gilles . The global median value of all measurements is given by $G=(6.674252^{+0.003655}_{-0.002342})\times 10^{-11}\mathrm{Nm^{2}kg^{-2}}$, where the error bars correspond to 95% c.l. We now group the measurements, similar to what was done for $H_{0}$ according to how the measurements were made. For our grouping criterion, we have considered the device used for measurement and the mode in which the device was used. In case of torsion balance, the same setup can be used for different procedures. Similar to $H_{0}$, we catagorize the grouping done according to mode and device as primary and secondary respectively.

A tabular summary of the median of $G$ measurements after collating the observations into various groups can be found in Table 11 and Table 11. Table 11 shows the results when classification of all $G$ measurements is done only by device. Similar grouping and tabulation are done for mode in Table 11. The median values after excluding all measurements of device and mode type are shown in Table 11 and Table 11 respectively. The sub-group median values of both the primary and secondary types are shown in  Table 12. The median estimate after grouping according to primary type is given by $G=6.674702\times 10^{-11}~{}\mathrm{Nm^{2}/kg^{2}}$, and after grouping by secondary type is equal to $6.673765\times 10^{-11}~{}\mathrm{Nm^{2}/kg^{2}}$. Unlike $H_{0}$, we could not obtain 95% c.l. intervals for the subgroup medians, since the total number of measurements were quite small. So we only calculate 68% c.l. uncertainties for the sub-group medians for $G$, which can be found in  Table 12.

5 Conclusions

In this article, we have described the usage of median statistics technique used to obtain the central values of parameters along with associated 95% c.l. uncertainties. We have demonstrated its robustness over mean statistics using numerical experiments. We have extended previous median statistics measurements by R11 to the full list of 576 Hubble Constant ($H_{0}$) measurements (updated as of Sept. 2016). We grouped all measurements according to primary and secondary categories (using the same classification as R11) and estimated the median value in each category. We then calculated the median of all these sub-group medians in both the categories. We find that for the primary type of measurements, the sub-group median estimate is given by $H_{0}=72.5\ \mathrm{kms^{-1}Mpc^{-1}}$ with $95\%$ confidence limits between $64.5$ and $75.0$ km/s/Mpc. We then carried out the same exercise for all measurements of Newton’s Gravitational Constant ($G$), tabulated since 1980. Here, the grouping was done on basis of the device used for measurements and the mode of measurement (for any given device). We find that the sub-group median (after splitting according to mode of measurement) is given by $G=6.674702\times 10^{-11}~{}\mathrm{Nm^{2}kg^{-2}}$ and $68\%$ confidence limits are $[6.671910,6.675565]\times 10^{-11}\mathrm{Nm^{2}kg^{-2}}$. The summary statistics of $H_{0}$ and $G$ are tabulated in  Table 12.

Acknowledgements.

Acknowledgements: We would like to thank Gang Chen for sharing with us the data used in R11.