Proactive Routing to Interpretable Surrogates with Distribution-Free Safety Guarantees

TL;DR

This paper introduces a distribution-free, input-based routing method that guarantees controlled model degradation, enabling safe and efficient use of interpretable surrogates alongside black-box models in diverse deployment scenarios.

Contribution

It proposes a novel proactive routing approach with conformal calibration that guarantees safety constraints and improves coverage over existing methods.

Findings

Maintains controlled violation rates across 35 datasets.

Achieves higher coverage than baseline routing methods.

Distribution-free safety guarantees are effective in practice.

Abstract

Model routing determines whether to use an accurate black-box model or a simpler surrogate that approximates it at lower cost or greater interpretability. In deployment settings, practitioners often wish to restrict surrogate use to inputs where its degradation relative to a reference model is controlled. We study proactive (input-based) routing, in which a lightweight gate selects the model before either runs, enabling distribution-free control of the fraction of routed inputs whose degradation exceeds a tolerance {\tau}. The gate is trained to distinguish safe from unsafe inputs, and a routing threshold is chosen via Clopper-Pearson conformal calibration on a held-out set, guaranteeing that the routed-set violation rate is at most {\alpha} with probability 1-{\delta}. We derive a feasibility condition linking safe routing to the base safe rate {\pi} and risk budget {\alpha}, along with sufficient AUC thresholds ensuring that feasible routing exists. Across 35 OpenML datasets and multiple black-box model families, gate-based conformal routing maintains controlled violation while achieving substantially higher coverage than regression conformal and naive baselines. We further show that probabilistic calibration primarily affects routing efficiency rather than distribution-free validity.