An Equal-Probability Partition of the Sample Space: A Non-parametric Inference from Finite Samples

TL;DR

This paper introduces a non-parametric method based on equal-probability partitions of the sample space, derived from order statistics, providing a new way to infer properties of continuous distributions from finite samples.

Contribution

It establishes that the sorted sample points create equal-probability segments with a fixed expected probability, offering a novel inference framework independent of distribution shape.

Findings

Partition segments each have an expected probability of 1/(N+1)

The entropy of the partition is log2(N+1) bits, quantifying information gain

Framework offers robust non-parametric inference for density and tail estimation

Abstract

This paper investigates what can be inferred about an arbitrary continuous probability distribution from a finite sample of $N$ observations drawn from it. The central finding is that the $N$ sorted sample points partition the real line into $N+1$ segments, each carrying an expected probability mass of exactly $1/(N+1)$. This non-parametric result, which follows from fundamental properties of order statistics, holds regardless of the underlying distribution's shape. This equal-probability partition yields a discrete entropy of $\log_2(N+1)$ bits, which quantifies the information gained from the sample and contrasts with Shannon's results for continuous variables. I compare this partition-based framework to the conventional ECDF and discuss its implications for robust non-parametric inference, particularly in density and tail estimation.