A Formal Approach to the Problem of Logical Non-Omniscience

TL;DR

This paper introduces the logical induction criterion for algorithms that assign and refine probabilities to logical statements over time, ensuring rational belief updates and desirable properties in logical reasoning systems.

Contribution

It formalizes a logical induction framework that guarantees rational probabilistic reasoning for computational agents over logical statements.

Findings

Logical inductors outperform their deductive processes.

They perform universal empirical induction given sufficient time.

They exhibit strong self-trust in their reasoning.

Abstract

We present the logical induction criterion for computable algorithms that assign probabilities to every logical statement in a given formal language, and refine those probabilities over time. The criterion is motivated by a series of stock trading analogies. Roughly speaking, each logical sentence phi is associated with a stock that is worth $1 per share if phi is true and nothing otherwise, and we interpret the belief-state of a logically uncertain reasoner as a set of market prices, where pt_N(phi)=50% means that on day N, shares of phi may be bought or sold from the reasoner for 50%. A market is then called a logical inductor if (very roughly) there is no polynomial-time computable trading strategy with finite risk tolerance that earns unbounded profits in that market over time. We then describe how this single criterion implies a number of desirable properties of bounded reasoners; for example, logical inductors outpace their underlying deductive process, perform universal empirical induction given enough time to think, and place strong trust in their own reasoning process.