# Estimating Feature-Label Dependence Using Gini Distance Statistics

## TL;DR

This paper introduces Gini distance statistics for dependence testing between features and labels, offering faster convergence and tighter error bounds than existing methods, with extensive experiments validating its effectiveness.

## Contribution

It proposes Gini distance covariance and correlation measures that characterize independence, along with theoretical bounds and empirical validation, advancing dependence testing in supervised learning.

## Key findings

- Gini distance statistics converge faster than distance covariance.
- The proposed measures provide tighter bounds on Type I and II errors.
- Experimental results demonstrate improved performance over existing methods.

## Abstract

Identifying statistical dependence between the features and the label is a fundamental problem in supervised learning. This paper presents a framework for estimating dependence between numerical features and a categorical label using generalized Gini distance, an energy distance in reproducing kernel Hilbert spaces (RKHS). Two Gini distance based dependence measures are explored: Gini distance covariance and Gini distance correlation. Unlike Pearson covariance and correlation, which do not characterize independence, the above Gini distance based measures define dependence as well as independence of random variables. The test statistics are simple to calculate and do not require probability density estimation. Uniform convergence bounds and asymptotic bounds are derived for the test statistics. Comparisons with distance covariance statistics are provided. It is shown that Gini distance statistics converge faster than distance covariance statistics in the uniform convergence bounds, hence tighter upper bounds on both Type I and Type II errors. Moreover, the probability of Gini distance covariance statistic under-performing the distance covariance statistic in Type II error decreases to 0 exponentially with the increase of the sample size. Extensive experimental results are presented to demonstrate the performance of the proposed method.