RS-Reg: Probabilistic and Robust Certified Regression Through Randomized Smoothing

TL;DR

This paper extends randomized smoothing techniques to regression tasks, establishing probabilistic robustness bounds against input perturbations and analyzing the effectiveness of simple smoothing functions.

Contribution

It introduces a flexible probabilistic robustness framework for regression, deriving bounds for bounded output models and analyzing averaging functions' asymptotic properties.

Findings

Certified robustness bounds for regression under input perturbations

Asymptotic properties of averaging functions in regression

Simulation results validating theoretical bounds

Abstract

Randomized smoothing has shown promising certified robustness against adversaries in classification tasks. Despite such success with only zeroth-order access to base models, randomized smoothing has not been extended to a general form of regression. By defining robustness in regression tasks flexibly through probabilities, we demonstrate how to establish upper bounds on input data point perturbation (using the $\ell_2$ norm) for a user-specified probability of observing valid outputs. Furthermore, we showcase the asymptotic property of a basic averaging function in scenarios where the regression model operates without any constraint. We then derive a certified upper bound of the input perturbations when dealing with a family of regression models where the outputs are bounded. Our simulations verify the validity of the theoretical results and reveal the advantages and limitations of simple smoothing functions, i.e., averaging, in regression tasks. The code is publicly available at \url{https://github.com/arekavandi/Certified_Robust_Regression}.