The pilgrim process

TL;DR

The paper introduces the pilgrim process, a probabilistic model for time-to-event data with connections to survival analysis, exchangeable partitions, and well-known stochastic processes, revealing new links between these concepts.

Contribution

It presents the pilgrim process as a novel, simple, and analytically tractable model linking survival analysis, exchangeable partitions, and classical stochastic processes.

Findings

Close connection with Kaplan-Meier estimator

Analytical results on the distribution of blocks

Links to Chinese restaurant and Indian buffet processes

Abstract

Pilgrim's monopoly is a probabilistic process giving rise to a non-negative sequence $T_1, T_2,\ldots$ that is infinitely exchangeable, a natural model for time-to-event data. The one-dimensional marginal distributions are exponential. The rules are simple, the process is easy to generate sequentially, and a simple expression is available for both the joint density and the multivariate survivor function. There is a close connection with the Kaplan-Meier estimator of the survival distribution. Embedded within the process is an infinitely exchangeable ordered partition processes connected to Markov branching processes in neutral evolutionary theory. Some aspects of the process, such as the distribution of the number of blocks, can be investigated analytically and confirmed by simulation. By ignoring the order, the embedded process can be considered as an infinitely exchangeable partition process, shown to be closely related to the Chinese restaurant process. Further connection to the Indian buffet process is also provided. Thus we establish a previously unknown link between the well-known Kaplan-Meier estimator and the important Ewens sampling formula.