Kendall transformation: a robust representation of continuous data for information theory

TL;DR

Kendall transformation converts continuous data into a categorical ranking form, enabling robust information-theoretic analysis especially with small sample sizes, by preserving order relations and simplifying complex interactions.

Contribution

The paper introduces Kendall transformation as a novel method for representing continuous data in a categorical form suitable for information theory, improving robustness and applicability in small-sample scenarios.

Findings

Enables direct application of information theory to continuous data without discretisation.

Improves robustness by focusing on ranking information, reducing sensitivity to outliers.

Demonstrates effectiveness in multivariate and real-world data analysis.

Abstract

Kendall transformation is a conversion of an ordered feature into a vector of pairwise order relations between individual values. This way, it preserves ranking of observations and represents it in a categorical form. Such transformation allows for generalisation of methods requiring strictly categorical input, especially in the limit of small number of observations, when discretisation becomes problematic. In particular, many approaches of information theory can be directly applied to Kendall-transformed continuous data without relying on differential entropy or any additional parameters. Moreover, by filtering information to this contained in ranking, Kendall transformation leads to a better robustness at a reasonable cost of dropping sophisticated interactions which are anyhow unlikely to be correctly estimated. In bivariate analysis, Kendall transformation can be related to popular non-parametric methods, showing the soundness of the approach. The paper also demonstrates its efficiency in multivariate problems, as well as provides an example analysis of a real-world data.