A Natural Law of Succession

TL;DR

This paper introduces a novel method for estimating the probability of the next symbol in a sequence based on past counts, outperforming existing methods in theory and practice.

Contribution

A new solution to multinomial estimation that improves accuracy over standard approaches, supported by theoretical analysis and practical experiments.

Findings

Outperforms standard estimation methods

Provides theoretical guarantees of improved accuracy

Demonstrates superior performance in practical tests

Abstract

Consider the problem of multinomial estimation. You are given an alphabet of k distinct symbols and are told that the i-th symbol occurred exactly n_i times in the past. On the basis of this information alone, you must now estimate the conditional probability that the next symbol will be i. In this report, we present a new solution to this fundamental problem in statistics and demonstrate that our solution outperforms standard approaches, both in theory and in practice.