GPS Information and Rate Tolerance - Clarifying Relationship between Rate Distortion and Complexity Distortion

TL;DR

This paper clarifies the relationship between rate distortion and complexity distortion, proposing that complexity distortion is a special case of rate tolerance, and introduces generalized information formulas using GPS as an example.

Contribution

It demonstrates that complexity distortion is a subset of rate tolerance and develops generalized mutual information formulas linking these concepts.

Findings

Complexity distortion is a special case of rate tolerance.

Rate distortion and rate tolerance can be described by a unified mutual information formula.

GPS example illustrates the generalized information measure.

Abstract

I proposed rate tolerance and discussed its relation to rate distortion in my book "A Generalized Information Theory" published in 1993. Recently, I examined the structure function and the complexity distortion based on Kolmogorov's complexity theory. It is my understanding now that complexity-distortion is only a special case of rate tolerance while constraint sets change from fuzzy sets into clear sets that look like balls with the same radius. It is not true that the complexity distortion is generally equivalent to rate distortion as claimed by the researchers of complexity theory. I conclude that a rate distortion function can only be equivalent to a rate tolerance function and both of them can be described by a generalized mutual information formula where P(Y|X) is equal to P(Y|Tolerance). The paper uses GPS as an example to derive generalized information formulae and proves the above conclusions using mathematical analyses and a coding example. The similarity between the formula for measuring GPS information and the formula for rate distortion function can deepen our understanding the generalized information measure.