Entropy estimation in bidimensional sequences

TL;DR

This paper compares entropy estimation methods for bidimensional sequences, finding that traditional block-entropies provide the most accurate estimates, while compression-based methods vary in effectiveness depending on their handling of correlations.

Contribution

It introduces a validation dataset of natural images and evaluates different entropy estimation techniques, highlighting the superiority of block-entropies and proposing a compressor-based symmetry detection method.

Findings

Block-entropies yield the best asymptotic entropy estimates.

Compression methods that preserve correlations perform better.

A compressor-based approach can detect symmetries in textures and images.

Abstract

We investigate the performance of entropy estimation methods, based either on block entropies or compression approaches, in the case of bidimensional sequences. We introduce a validation dataset made of images produced by a large number of different natural systems, in the vast majority characterized by long-range correlations, which produce a large spectrum of entropies. Results show that the framework based on lossless compressors applied to the one-dimensional projection of the considered dataset leads to poor estimates. This is because higher-dimensional correlations are lost in the projection operation. The adoption of compression methods which do not introduce dimensionality reduction improves the performance of this approach. By far, the best estimation of the asymptotic entropy is generated by the faster convergence of the traditional block-entropies method. As a by-product of our analysis, we show how a specific compressor method can be used as a potentially interesting technique for automatic detection of symmetries in textures and images.