Decision Support under Prediction-Induced Censoring

TL;DR

This paper introduces PIC-RL, an adaptive reinforcement learning framework that addresses prediction-induced censoring in large-scale resource allocation, improving demand estimation and decision-making in generative AI serving systems.

Contribution

The paper proposes a novel closed-loop reinforcement learning approach that transforms censoring issues into decision signals, with theoretical guarantees and practical validation on Alibaba GenAI data.

Findings

PIC-RL reduces service degradation by up to 50%

It outperforms state-of-the-art baselines in demand prediction and resource allocation

The framework maintains cost efficiency while improving system robustness

Abstract

In many data-driven online decision systems, actions determine not only operational costs but also the data availability for future learning -- a phenomenon termed Prediction-Induced Censoring (PIC). This challenge is particularly acute in large-scale resource allocation for generative AI (GenAI) serving: insufficient capacity triggers shortages but hides the true demand, leaving the system with only a "greater-than" constraint. Standard decision-making approaches that rely on uncensored data suffer from selection bias, often locking the system into a self-reinforcing low-provisioning trap. To break this loop, this paper proposes an adaptive approach named PIC-Reinforcement Learning (PIC-RL), a closed-loop framework that transforms censoring from a data quality problem into a decision signal. PIC-RL integrates (1) Uncertainty-Aware Demand Prediction to manage the information-cost trade-off, (2) Pessimistic Surrogate Inference to construct decision-aligned conservative feedback from shortage events, and (3) Dual-Timescale Adaptation to stabilize online learning against distribution drift. The analysis provides theoretical guarantees that the feedback design corrects the selection bias inherent in naive learning. Experiments on production Alibaba GenAI traces demonstrate that PIC-RL consistently outperforms state-of-the-art baselines, reducing service degradation by up to 50% while maintaining cost efficiency.