Smoothed Embeddings for Certified Few-Shot Learning

TL;DR

This paper extends randomized smoothing to few-shot learning models that use normalized embeddings, providing robustness certificates against $ ext{l}_2$-bounded perturbations, supported by theoretical analysis and experimental validation.

Contribution

It introduces a novel application of randomized smoothing to embedding-based few-shot learning models, including robustness analysis and certification methods.

Findings

Robustness certificates against $ ext{l}_2$ perturbations for embedding models

Theoretical analysis of Lipschitz continuity in such models

Experimental validation on multiple datasets

Abstract

Randomized smoothing is considered to be the state-of-the-art provable defense against adversarial perturbations. However, it heavily exploits the fact that classifiers map input objects to class probabilities and do not focus on the ones that learn a metric space in which classification is performed by computing distances to embeddings of classes prototypes. In this work, we extend randomized smoothing to few-shot learning models that map inputs to normalized embeddings. We provide analysis of Lipschitz continuity of such models and derive robustness certificate against $\ell_2$-bounded perturbations that may be useful in few-shot learning scenarios. Our theoretical results are confirmed by experiments on different datasets.