Distance Correlation Methods for Discovering Associations in Large Astrophysical Databases

TL;DR

This paper demonstrates that the distance correlation coefficient is a powerful tool for detecting complex, nonlinear associations in large astrophysical datasets, outperforming traditional methods like Pearson and maximal information coefficient.

Contribution

It introduces the application of the distance correlation coefficient to astrophysical data, showing its advantages in identifying nonlinear and independent relationships in large galaxy databases.

Findings

Distance correlation detects more associations than Pearson.

It outperforms maximal information coefficient in resolving data patterns.

Applicable across various galaxy types and redshifts.

Abstract

High-dimensional, large-sample astrophysical databases of galaxy clusters, such as the Chandra Deep Field South COMBO-17 database, provide measurements on many variables for thousands of galaxies and a range of redshifts. Current understanding of galaxy formation and evolution rests sensitively on relationships between different astrophysical variables; hence an ability to detect and verify associations or correlations between variables is important in astrophysical research. In this paper, we apply a recently defined statistical measure called the distance correlation coefficient which can be used to identify new associations and correlations between astrophysical variables. The distance correlation coefficient applies to variables of any dimension; it can be used to determine smaller sets of variables that provide equivalent astrophysical information; it is zero only when variables are independent; and it is capable of detecting nonlinear associations that are undetectable by the classical Pearson correlation coefficient. Hence, the distance correlation coefficient provides more information than the Pearson coefficient. We analyze numerous pairs of variables in the COMBO-17 database with the distance correlation method and with the maximal information coefficient. We show that the Pearson coefficient can be estimated with higher accuracy from the corresponding distance correlation coefficient than from the maximal information coefficient. For given values of the Pearson coefficient, the distance correlation method has a greater ability than the maximal information coefficient to resolve astrophysical data into highly concentrated V-shapes, which enhances classification and pattern identification. These results are observed over a range of redshifts beyond the local universe and for galaxies from elliptical to spiral.