Worst-Case Discovery and Runtime Protection for RL-Based Network Controllers

TL;DR

ReGuard is a framework that identifies worst-case scenarios for RL-based network controllers and applies lightweight, rule-based interventions at inference time to mitigate performance degradation without retraining.

Contribution

It introduces a bilevel regret-maximization approach to discover worst-case scenarios and develops a lightweight rule-based protection method for RL controllers.

Findings

ReGuard discovers scenarios where controllers perform 43-64% worse than optimal.

It finds larger performance gaps than existing baselines, up to 6 times bigger.

Protection via ReGuard reduces worst-case gaps by 79-85%, improving overall performance.

Abstract

RL-based controllers achieve strong average-case performance in networking tasks such as congestion control and adaptive bitrate streaming. Yet their performance can degrade severely under network conditions where strong performance is still achievable. Identifying such conditions and quantifying the resulting performance gap is intractable by enumeration, while the sequential and closed-loop nature of RL controllers makes formal verification methods impractical. We present ReGuard, a framework that discovers worst-case scenarios for a given RL controller and protects it against them at inference time without retraining. Discovery is formulated as a bilevel regret-maximization problem, which yields a certified lower bound on the worst-case performance gap. The discovered trajectories are then analyzed as counterfactuals and compiled into lightweight logic rules that intervene only when a risky state is detected, leaving the controller's behavior unchanged otherwise. We evaluate ReGuard across three RL-based network controllers: Pensieve, Sage, and Park. ReGuard discovers scenarios in which the controller's performance is 43$-$64% worse than what is achievable. ReGuard not only discovers gaps 57% to 6$\times$ larger than those found by the strongest baselines but also shrinks them by 79$-$85% via lightweight rule-based protection while preserving nominal performance. ReGuard's protection extends beyond the scenarios it discovers, improving performance across a wider range of network conditions.