Dependence and Uncertainty: Information Measures using Tsallis Entropy

TL;DR

This paper introduces a novel framework for quantifying dependence and uncertainty in multivariate data using Tsallis entropy within the copula-based approach, addressing limitations of existing measures and demonstrating practical applications.

Contribution

It extends copula-based information measures by incorporating Tsallis entropy, develops a non-parametric version, and proposes a new mutual information measure for testing independence.

Findings

The proposed cumulative copula Tsallis entropy has well-defined properties and bounds.

The non-parametric measure effectively captures dependence in complex systems.

The new mutual information measure outperforms traditional density-based methods in independence testing.

Abstract

In multivariate analysis, uncertainty arises from two sources: the marginal distributions of the variables and their dependence structure. Quantifying the dependence structure is crucial, as it provides valuable insights into the relationships among components of a random vector. Copula functions effectively capture this dependence structure independent of marginals, making copula-based information measures highly significant. However, existing copula-based information measures, such as entropy, divergence, and mutual information, rely on copula densities, which may not exist in many scenarios, limiting their applicability. Recently, to address this issue, Arshad et al. (2024) introduced cumulative copula-based measures using Shannon entropy. In this paper, we extend this framework by using Tsallis entropy, a non-additive entropy that provides greater flexibility for quantifying uncertainties. We propose cumulative copula Tsallis entropy, derive its properties and bounds, and illustrate its utility through examples. We further develop a non-parametric version of the measure and validate it using coupled periodic and chaotic maps. Additionally, we extend Kerridge's inaccuracy measure and Kullback-Leibler (KL) divergence to the cumulative copula framework. Using the relationship between KL divergence and mutual information, we propose a new cumulative mutual information (CMI) measure, which outperform the limitations of density-based mutual information. Furthermore, we introduce a test procedure for testing the mutual independence among random variables using CMI measure. Finally, we illustrate the potential of the proposed CMI measure as an economic indicator through real bivariate financial time series data.