Reliability-Aware Determinantal Point Processes for Robust Informative Data Selection in Large Language Models

TL;DR

This paper introduces ProbDPP, a reliability-aware determinantal point process method for robust, diverse data selection in large language models, addressing data access uncertainties with a new objective and online learning algorithm.

Contribution

It proposes ProbDPP, a novel approach that incorporates data reliability into DPP-based selection, along with an online learning algorithm with theoretical guarantees.

Findings

ProbDPP effectively accounts for data access unreliability.

The proposed algorithm achieves bounded regret in online learning.

The method enhances data selection robustness under uncertainty.

Abstract

Informative data selection is a key requirement for large language models (LLMs) to minimize the amount of data required for fine-tuning, network distillation, and token pruning, enabling fast and efficient deployment, especially under computational and communication constraints. Traditional subset selection methods, including those based on Determinantal Point Processes (DPP), focus on maximizing diversity but assume that selected data batches are always available error-free. This presumption prohibits their use under partial storage outage, imperfect communication, and stochastic access failures. Furthermore, we show that the original formulation collapses under such conditions. To address this gap, we introduce ProbDPP, a novel reliability-aware implementation of k-DPP that accounts for probabilistic data access by recasting the objective function with a regularization term that remains well-posed and decomposes into a geometric diversity term and unreliability cost. The resulting objective facilitates robust selection of diverse data batches under uncertainty. Furthermore, we frame this reliability-aware diversity maximization as a combinatorial semi-bandit problem and propose a UCB-style algorithm to efficiently learn the unknown reliability online. Theoretical analysis provides regret bounds for the proposed approach, ensuring performance guarantees.