Description Complexity of Unary Structures in First-Order Logic with Links to Entropy

TL;DR

This paper investigates the description complexity of unary structures in first-order logic, establishing asymptotic bounds and linking it to entropy measures like Shannon and Boltzmann entropy.

Contribution

It provides tight asymptotic bounds for description complexity of unary structures and connects logical complexity with entropy concepts.

Findings

Upper and lower bounds for description complexity match asymptotically.

Established a relationship between Shannon entropy and description complexity.

Extended the entropy connection to Boltzmann entropy.

Abstract

The description complexity of a model is the length of the shortest formula that defines the model. We study the description complexity of unary structures in first-order logic FO, also drawing links to semantic complexity in the form of entropy. The class of unary structures provides, e.g., a simple way to represent tabular Boolean data sets as relational structures. We define structures with FO-formulas that are strictly linear in the size of the model as opposed to using the naive quadratic ones, and we use arguments based on formula size games to obtain related lower bounds for description complexity. For a typical structure the upper and lower bounds in fact match up to a sublinear term, leading to a precise asymptotic result on the expected description complexity of a randomly selected structure. We then give bounds on the relationship between Shannon entropy and description complexity. We extend this relationship also to Boltzmann entropy by establishing an asymptotic match between the two entropies. Despite the simplicity of unary structures, our arguments require the use of formula size games, Stirling's approximation and Chernoff bounds.