Shannon entropy estimation for linear processes

TL;DR

This paper introduces an estimator for Shannon entropy of linear processes using kernel density estimation, proving its almost sure and L^2 convergence under certain conditions.

Contribution

It presents a new entropy estimator for linear processes and establishes its convergence properties, advancing the statistical analysis of such processes.

Findings

The estimator converges almost surely to the true entropy.

The estimator converges in L^2 norm.

Conditions for convergence are specified.

Abstract

In this paper, we estimate the Shannon entropy $S(f) = -\E[ \log (f(x))]$ of a one-sided linear process with probability density function $f(x)$. We employ the integral estimator $S_n(f)$, which utilizes the standard kernel density estimator $f_n(x)$ of $f(x)$. We show that $S_n (f)$ converges to $S(f)$ almost surely and in $\L^2$ under reasonable conditions.