Extropy Rate: Properties and Application in Feature Selection

TL;DR

This paper introduces the concept of extropy rate for stochastic processes, explores its properties, and demonstrates its application in feature selection, showing it effectively captures information and improves performance on real datasets.

Contribution

The study defines extropy rate for stochastic processes, analyzes its properties, and applies it to develop a novel feature selection method that outperforms existing approaches.

Findings

Extropy rate effectively captures information in time series and dynamical systems.

The proposed feature selection method outperforms existing methods on six datasets.

Extropy rate correlates closely with diversity indices like Simpson's index.

Abstract

Extropy, a complementary dual of entropy, (proposed by Lad et al. \cite{lad2015extropy} in 2015) has attracted considerable interest from the research community. In this study, we focus on discrete random variables and define conditional extropy, establishing key properties of joint and conditional extropy such as bounds, uncertainty reduction due to additional information, and Lipschitz continuity. We further introduce the concept of extropy rate for a stochastic process of discrete random variables as a measure of the average uncertainty per random variable within the process. It is observed that for infinite stationary and ergodic stochastic processes, as well as for identically and independently distributed sequences, the extropy rate exhibits asymptotic equivalence. We explore the extropy rate for finite stochastic processes and numerically illustrate its effectiveness in capturing the underlying information across various distributions, quantifying complexity in time series data, and characterizing chaotic dynamics in dynamical systems. The behaviour of estimated extropy rate is observed to be closely aligned with Simpson's diversity index. The real-life applicability of the extropy rate is presented through a novel feature selection method based on the fact that features with higher extropy rates contain greater inherent information. Using six publicly available datasets, we show the superiority of the proposed feature selection method over some other existing popular approaches.