Breaking through the classical Shannon entropy limit: A new frontier through logical semantics

TL;DR

This paper introduces a Shannon-style analysis of communication systems with deductive reasoning, showing that incorporating semantics through logic can significantly improve communication efficiency beyond classical limits.

Contribution

It is the first to analyze a semantic communication system with logical inference using information theory, demonstrating potential efficiency gains.

Findings

Deductive reasoning enhances communication efficiency.

Logical inference can surpass classical Shannon limits.

Practical codes illustrate the benefits of semantics in communication.

Abstract

Information theory has provided foundations for the theories of several application areas critical for modern society, including communications, computer storage, and AI. A key aspect of Shannon's 1948 theory is a sharp lower bound on the number of bits needed to encode and communicate a string of symbols. When he introduced the theory, Shannon famously excluded any notion of semantics behind the symbols being communicated. This semantics-free notion went on to have massive impact on communication and computing technologies, even as multiple proposals for reintroducing semantics in a theory of information were being made, notably one where Carnap and Bar-Hillel used logic and reasoning to capture semantics. In this paper we present, for the first time, a Shannon-style analysis of a communication system equipped with a deductive reasoning capability, implemented using logical inference. We use some of the most important techniques developed in information theory to demonstrate significant and sometimes surprising gains in communication efficiency availed to us through such capability, demonstrated also through practical codes. We thus argue that proposals for a semantic information theory should include the power of deductive reasoning to magnify the value of transmitted bits as we strive to fully unlock the inherent potential of semantics.