Information Measure as Time Complexity

TL;DR

This paper introduces an information-theoretic approach to analyze the time complexity of search problems, linking entropy and mutual information to computational difficulty and providing insights into P vs. NP and related complexity classes.

Contribution

It generalizes Shannon entropy to functions, establishing a novel connection between information measures and computational complexity, and offers a new method to approach P vs. NP problem.

Findings

Information measure relates to search problem complexity.

Supports P=RP=BPP and P!=PP conjectures.

Proposes entropy-based lower bounds on query complexity.

Abstract

The concept of Shannon entropy of random variables was generalized to measurable functions in general, and to simple functions with finite values in particular. It is shown that the information measure of a function is related to the time complexity of search problems concerning the functions in question. Formally, given a Turing reduction from a search problem f(x)=y to another function g(x), the amount of information about f(x)=y provided by querying g(x) is exactly equal to the average mutual information I(f;g). As a result, the average number of queries is I(f=y)/I(f;g), where I(f) is amount of self-information about the event {f=y}. In the idea case, if I(f=y)/I(f;g) is polynomial in the size of input and the function g(x) can be computed in polynomial time, then the problem f(x)=y also has polynomial-time algorithm. As it turns out, our information-based complexity estimation is a natural setting in which to study the power of randomized or probabilistic algorithms. Applying to decision problems, our result provides a strong evidence that P=RP=BPP. Using brute-force search as a benchmark for efficiency, our results also support that P!=PP. Further, applying an argument similar to Carnot's work on measuring the efficiency of an ideal heat engine in thermodynamics, it is shown that I(f=y)/H(f) is a lower bound on query complexity of solving f(x)=y. According to Markov's inequality, if I(f=y)/H(f) is exponential in the size of input, then the probability of f(x)=y having polynomial-time algorithm is a negative exponential in the size of input. This result enables us to reduce the problem of proving FP!=FNP to the computation of the entropy of certain simple functions, such as the subset sum function. As a result, P!=NP is provable, answering a long-standing open problem.