Smoothing the Black-Box: Signed-Distance Supervision for Black-Box Model Copying

TL;DR

This paper introduces a novel distance-based framework for black-box model copying that improves boundary reconstruction by replacing hard-label supervision with signed distances, leading to better fidelity and uncertainty estimation.

Contribution

It proposes a smoothing and regularization scheme using signed distances to enhance black-box copying, along with two algorithms for distance estimation with label-only access.

Findings

Improved fidelity and generalization accuracy over hard-label methods

Enhanced boundary reconstruction through signed-distance supervision

Distance outputs serve as uncertainty signals for black-box models

Abstract

Deployed machine learning systems must continuously evolve as data, architectures, and regulations change, often without access to original training data or model internals. In such settings, black-box copying provides a practical refactoring mechanism, i.e. upgrading legacy models by learning replicas from input-output queries alone. When restricted to hard-label outputs, copying turns into a discontinuous surface reconstruction problem from pointwise queries, severely limiting the ability to recover boundary geometry efficiently. We propose a distance-based copying (distillation) framework that replaces hard-label supervision with signed distances to the teacher's decision boundary, converting copying into a smooth regression problem that exploits local geometry. We develop an $\alpha$-governed smoothing and regularization scheme with H\"older/Lipschitz control over the induced target surface, and introduce two model-agnostic algorithms to estimate signed distances under label-only access. Experiments on synthetic problems and UCI benchmarks show consistent improvements in fidelity and generalization accuracy over hard-label baselines, while enabling distance outputs as uncertainty-related signals for black-box replicas.