Semantic Identity Compression: Zero-Error Laws, Rate-Distortion, and Neurosymbolic Necessity

TL;DR

This paper establishes fundamental limits and laws governing semantic identity compression in neural and symbolic systems, emphasizing the importance of collision-fiber geometry for exact identity recovery.

Contribution

It introduces finite laws and rate-distortion principles based on collision-fiber geometry, linking structural ambiguity to symbolic identity mechanisms.

Findings

Derived fixed-length converse law for identity recovery

Established exact fiberwise rate-distortion law for finite sources

Connected collision-fiber geometry to query complexity and system structure

Abstract

Symbolic systems operate over precise identities: variables denote specific objects, pointers target precise memory locations, and database keys refer to singular records. Neural embeddings generalize by compressing away semantic detail, but this compression creates collision ambiguity: multiple distinct entities can share the same representation value. Exact identity recovery requires additional information precisely when representation fibers have size greater than one. The residual cost is controlled by a single combinatorial object: the collision-fiber geometry of the representation map $\pi$. Let $A_{\pi}=\max_u |\pi^{-1}(u)|$ be the largest collision fiber. The finite laws include a tight fixed-length converse $L \ge \log_2 A_{\pi}$, an exact finite-block scaling law, a pointwise adaptive budget $\lceil \log_2 |\pi^{-1}(u)|\rceil$, and an exact fiberwise rate-distortion law for arbitrary finite sources via recoverable-mass decomposition across representation fibers. The uniform single-block formula $D^\star(L)=\max(0,1-2^L/a)$ appears as a closed-form special case when all mass lies on one collision block, where $a = A_{\pi}$ is the collision block size. The same fiber geometry determines query complexity and canonical structure for distinguishing families. Because this residual ambiguity is structural rather than representation-specific, symbolic identity mechanisms (handles, keys, pointers, nominal tags) are the necessary system-level complement to any non-injective semantic representation. All main results are machine-checked in Lean 4.