AdaptDel: Adaptable Deletion Rate Randomized Smoothing for Certified Robustness

TL;DR

AdaptDel introduces adaptable deletion rate randomized smoothing for sequence classification, significantly improving certified robustness against edit distance perturbations, especially for inputs of varying lengths in natural language processing.

Contribution

It extends randomized smoothing to variable deletion rates, enabling dynamic adjustment based on input properties for better robustness certification.

Findings

Up to 30 orders of magnitude improvement in median certified region size.

Strong empirical results on natural language tasks.

Theoretical soundness with respect to edit distance.

Abstract

We consider the problem of certified robustness for sequence classification against edit distance perturbations. Naturally occurring inputs of varying lengths (e.g., sentences in natural language processing tasks) present a challenge to current methods that employ fixed-rate deletion mechanisms and lead to suboptimal performance. To this end, we introduce AdaptDel methods with adaptable deletion rates that dynamically adjust based on input properties. We extend the theoretical framework of randomized smoothing to variable-rate deletion, ensuring sound certification with respect to edit distance. We achieve strong empirical results in natural language tasks, observing up to 30 orders of magnitude improvement to median cardinality of the certified region, over state-of-the-art certifications.