Kolmogorov Complexity in perspective. Part II: Classification, Information Processing and Duality

TL;DR

This paper explores the role of Kolmogorov complexity in classification and information systems, highlighting dualities between operational modes and their connections to logic, set theory, and data analysis.

Contribution

It unifies diverse approaches to classification using Kolmogorov complexity, emphasizing the duality between Bottom-Up and Top-Down methods and their relation to logic and set theory.

Findings

Classification using compression relates to Kolmogorov complexity.

Duality between Bottom-Up and Top-Down modes in information processing.

Connections between Kolmogorov complexity, logic, and set theory.

Abstract

We survey diverse approaches to the notion of information: from Shannon entropy to Kolmogorov complexity. Two of the main applications of Kolmogorov complexity are presented: randomness and classification. The survey is divided in two parts published in a same volume. Part II is dedicated to the relation between logic and information system, within the scope of Kolmogorov algorithmic information theory. We present a recent application of Kolmogorov complexity: classification using compression, an idea with provocative implementation by authors such as Bennett, Vitanyi and Cilibrasi. This stresses how Kolmogorov complexity, besides being a foundation to randomness, is also related to classification. Another approach to classification is also considered: the so-called "Google classification". It uses another original and attractive idea which is connected to the classification using compression and to Kolmogorov complexity from a conceptual point of view. We present and unify these different approaches to classification in terms of Bottom-Up versus Top-Down operational modes, of which we point the fundamental principles and the underlying duality. We look at the way these two dual modes are used in different approaches to information system, particularly the relational model for database introduced by Codd in the 70's. This allows to point out diverse forms of a fundamental duality. These operational modes are also reinterpreted in the context of the comprehension schema of axiomatic set theory ZF. This leads us to develop how Kolmogorov's complexity is linked to intensionality, abstraction, classification and information system.