DIPS: Optimal Dynamic Index for Poisson $\boldsymbol{\pi}$ps Sampling

TL;DR

This paper introduces a dynamic index for Poisson $oldsymbol{ ext{pi}}$ps sampling that supports constant-time queries and updates, enabling efficient handling of dynamic data with practical applications in data mining.

Contribution

We propose a novel dynamic index tailored for Poisson $ ext{pi}$ps sampling, achieving optimal expected query and update times with a recursive partitioning approach.

Findings

Supports $ ext{O}(1)$ expected query and update time

Achieves $ ext{O}(n)$ space complexity

Demonstrates significant speedups in empirical evaluations

Abstract

This paper addresses the Poisson $\pi$ps sampling problem, a topic of significant academic interest in various domains and with practical data mining applications, such as influence maximization. The problem includes a set $\mathcal{S}$ of $n$ elements, where each element $v$ is assigned a weight $w(v)$ reflecting its importance. The goal is to generate a random subset $X$ of $\mathcal{S}$, where each element $v \in \mathcal{S}$ is included in $X$ independently with probability $\frac{c\cdot w(v)}{\sum_{v \in \mathcal{S}} w(v)}$, where $0<c\leq 1$ is a constant. The subsets must be independent across different queries. While the Poisson $\pi$ps sampling problem can be reduced to the well-studied subset sampling problem, updates in Poisson $\pi$ps sampling, such as adding a new element or removing an element, would cause the probabilities of all $n$ elements to change in the corresponding subset sampling problem, making this approach impractical for dynamic scenarios. To address this, we propose a dynamic index specifically tailored for the Poisson $\pi$ps sampling problem, supporting optimal expected $\mathcal{O}(1)$ query time and $\mathcal{O}(1)$ index update time, with an optimal $\mathcal{O}(n)$ space cost. Our solution involves recursively partitioning the set by weights and ultimately using table lookup. The core of our solution lies in addressing the challenges posed by weight explosion and correlations between elements. Empirical evaluations demonstrate that our approach achieves significant speedups in update time while maintaining consistently competitive query time compared to the subset-sampling-based methods.