Triggered urn models for frequently asked questions (FAQ)

TL;DR

This paper introduces a generalized urn model with triggers for FAQ systems, providing a comprehensive asymptotic analysis that covers various special cases, revealing diverse limit distributions and practical implications.

Contribution

It develops a broad theoretical framework for triggered urn models, including time-dependent probabilities, and connects these to real-world FAQ data analysis.

Findings

Number of colors can remain finite or grow infinitely, depending on trigger probabilities.

Limit distributions range from normal to power-law, including Poisson and Zipf's law.

The theory aligns well with empirical data from Amazon sales and simulations.

Abstract

We investigate a nonclassic urn model with triggers that increase the number of colors. The scheme has emerged as a model for web services that set up frequently asked questions (FAQ). We present a thorough asymptotic analysis of the FAQ urn scheme in generality that covers a large number of special cases, such as Simon urn. For instance, we consider time dependent triggering probabilities. We identify regularity conditions on these probabilities that classify the schemes into those where the number of colors in the urn remains almost surely finite or increases to infinity and conditions that tell us whether all the existing colors are observed infinitely often or not. We determine the rank curve, too. In view of the broad generality of the trigger probabilities, a spectrum of limit distributions appears, from central limit theorems to Poisson approximation, to power-laws, revealing connections to Heap's exponent and Zipf's law. A combinatorial approach to the Simon urn is presented to indicate the possibility of such exact analysis, which is important for short-term predictions. Extensive simulations on real datasets (from Amazon sales) as well as computer-generated data clearly indicate that the asymptotic and exact theory developed agrees with practice.