Exact Bounds of Spearman's footrule in the Presence of Missing Data with Applications to Independence Testing

TL;DR

This paper derives exact bounds for Spearman's footrule with missing data, introduces efficient algorithms for these bounds, and applies them to develop a robust independence testing method that handles missing data effectively.

Contribution

It provides the first exact bounds for Spearman's footrule with missing data and introduces a new independence test that maintains Type I error control under arbitrary missingness.

Findings

Algorithms with $O(n^2)$ and $O(n^3)$ complexity for bounds computation.

Proposed independence test controls Type I error under arbitrary missingness.

Test demonstrates good power when missing data proportion is below 15%.

Abstract

This work studies exact bounds of Spearman's footrule between two partially observed $n$-dimensional distinct real-valued vectors $X$ and $Y$. The lower bound is obtained by sequentially constructing imputations of the partially observed vectors, each with a non-increasing value of Spearman's footrule. The upper bound is found by first considering the set of all possible values of Spearman's footrule for imputations of $X$ and $Y$, and then the size of this set is gradually reduced using several constraints. Algorithms with computational complexities $O(n^2)$ and $O(n^3)$ are provided for computing the lower and upper bound of Spearman's footrule for $X$ and $Y$, respectively. As an application of the bounds, we propose a novel two-sample independence testing method for data with missing values. Improving on all existing approaches, our method controls the Type I error under arbitrary missingness. Simulation results demonstrate our method has good power, typically when the proportion of pairs containing missing data is below $15\%$.