Adversarially Robust Kernel Smoothing

TL;DR

This paper introduces a scalable, robust learning algorithm that combines kernel smoothing with robust optimization, providing theoretical guarantees and competitive empirical performance under distribution shifts.

Contribution

It presents a novel robust learning method using kernel smoothing and integral operators, applicable to various models and loss functions, with theoretical and empirical validation.

Findings

The method offers certified robustness guarantees under distribution shift.

It achieves competitive performance with state-of-the-art certifiable robust algorithms.

The approach is applicable to deep neural networks and general loss functions.

Abstract

We propose a scalable robust learning algorithm combining kernel smoothing and robust optimization. Our method is motivated by the convex analysis perspective of distributionally robust optimization based on probability metrics, such as the Wasserstein distance and the maximum mean discrepancy. We adapt the integral operator using supremal convolution in convex analysis to form a novel function majorant used for enforcing robustness. Our method is simple in form and applies to general loss functions and machine learning models. Exploiting a connection with optimal transport, we prove theoretical guarantees for certified robustness under distribution shift. Furthermore, we report experiments with general machine learning models, such as deep neural networks, to demonstrate competitive performance with the state-of-the-art certifiable robust learning algorithms based on the Wasserstein distance.