Rate Distortion Theory for Descriptive Statistics

TL;DR

This paper explores how rate distortion theory can be applied to analyze complex statistical data, including outlier detection, optimal compression, and confidence region assignment, focusing on methodological aspects.

Contribution

It introduces a novel application of rate distortion theory to statistical analysis of complex data sets, emphasizing methodological development.

Findings

Effective outlier detection methods

Optimal compression rate determination

Construction of descriptive confidence regions

Abstract

Rate distortion theory was developed for optimizing lossy compression of data, but it also has a lot of applications in statistics. In this paper we will see how rate distortion theory can be used to analyze a complicated data set involving orientations of early Islamic mosques. The analysis involves testing, identification of outliers, choice of compression rate, calculation of optimal reconstruction points, and assigning "descriptive confidence regions" to the reconstruction points. In this paper the focus will be on the methods, so the integrity of the data set and the interpretation of the results will not be discussed.