Semantic Rate-Distortion Theory: Deductive Compression and Closure Fidelity

TL;DR

This paper develops a semantic rate-distortion theory for knowledge bases with closure fidelity, showing how redundancy affects rate and enabling more efficient communication by exploiting core structures.

Contribution

It introduces a novel semantic rate-distortion framework based on logical closure, revealing redundancy invisibility and a semantic leverage phenomenon for efficient knowledge transmission.

Findings

Semantic rate is below classical entropy when redundancy exists.

Full semantic rate depends only on the irredundant core, not redundant states.

Redundant states become free under closure fidelity, reducing required channel uses.

Abstract

Shannon's rate-distortion theory treats source symbols as unstructured labels. When the source is a knowledge base equipped with a logical proof system, a natural fidelity criterion is closure fidelity: a reconstruction is acceptable if it preserves the deductive closure of the original. This paper develops a rate-distortion theory under this criterion. Central to the theory is the irredundant core-a canonical generating set extracted by a fixed-order deletion procedure, from which the full deductive closure can be rederived. We prove that the zero-distortion semantic rate equals a quantity that is strictly below the classical entropy rate whenever the knowledge base contains redundant states. More generally, the full semantic rate-distortion function depends only on the core; redundant states are invisible to both rate and distortion. We derive a semantic source-channel separation theorem showing a semantic leverage phenomenon: under closure fidelity, the required source rate is reduced by an asymptotic leverage factor greater than one, allowing the same knowledge base to be communicated with proportionally fewer channel uses-not by violating Shannon capacity, but because redundant states become free. We also prove a strengthened Fano inequality that exploits core structure. For heterogeneous multi-agent communication, an overlap decomposition gives necessary and sufficient conditions for closure-reliable transmission and identifies a semantic bottleneck in broadcast settings that persists even over noiseless channels. All results are verified on Datalog instances with up to 24,000 base facts.