Adaptive Double-Booking Strategy for Outpatient Scheduling Using Multi-Objective Reinforcement Learning

TL;DR

This paper introduces an adaptive outpatient scheduling framework that uses multi-objective reinforcement learning and individualized no-show predictions to optimize booking strategies, reducing congestion and wait times.

Contribution

It presents a novel reinforcement learning approach incorporating patient-specific no-show risk and a co-evolution mechanism for policy diversity and convergence.

Findings

Improved scheduling decisions balancing congestion and patient wait times.

Enhanced policy convergence through the proposed { au} rule and co-evolution mechanism.

Effective interpretation of no-show risk and scheduling choices using SHAP explanations.

Abstract

Patient no-shows disrupt outpatient clinic operations, reduce productivity, and may delay necessary care. Clinics often adopt overbooking or double-booking to mitigate these effects. However, poorly calibrated policies can increase congestion and waiting times. Most existing methods rely on fixed heuristics and fail to adapt to real-time scheduling conditions or patient-specific no-show risk. To address these limitations, we propose an adaptive outpatient double-booking framework that integrates individualized no-show prediction with multi-objective reinforcement learning. The scheduling problem is formulated as a Markov decision process, and patient-level no-show probabilities estimated by a Multi-Head Attention Soft Random Forest model are incorporated in the reinforcement learning state. We develop a Multi-Policy Proximal Policy Optimization method equipped with a Multi-Policy Co-Evolution Mechanism. Under this mechanism, we propose a novel {\tau} rule based on Kullback-Leibler divergence that enables selective knowledge transfer among behaviorally similar policies, improving convergence and expanding the diversity of trade-offs. In addition, SHapley Additive exPlanations is used to interpret both the predicted no-show risk and the agent's scheduling decisions. The proposed framework determines when to single-book, double-book, or reject appointment requests, providing a dynamic and data-driven alternative to conventional outpatient scheduling policies.