Certified Robustness under Heterogeneous Perturbations via Hybrid Randomized Smoothing

TL;DR

This paper introduces a unified randomized smoothing framework for multimodal models that provides certified robustness against joint perturbations of discrete and continuous inputs, generalizing existing methods.

Contribution

It develops a Neyman--Pearson based smoothing approach that offers a closed-form robustness certificate for mixed discrete-continuous inputs, applicable to multimodal safety filtering.

Findings

First model-agnostic Neyman--Pearson certificate for joint discrete and continuous perturbations.

Validates the framework on multimodal safety filtering tasks.

Provides a strictly generalized robustness certificate encompassing Gaussian and discrete smoothing.

Abstract

Randomized smoothing provides strong, model-agnostic robustness certificates, but existing guarantees are limited to single modalities, treating continuous and discrete inputs in isolation. This limitation becomes critical in multimodal models, where decisions depend on cross-modal semantics and adversaries can jointly perturb heterogeneous inputs, rendering unimodal certificates insufficient. We introduce a unified randomized smoothing framework for mixed discrete--continuous inputs based on an analytically tractable Neyman--Pearson formulation of the joint worst-case problem. By analyzing the joint likelihood ordering induced by factorized discrete and continuous noise, our approach yields a closed-form, one-dimensional certificate that strictly generalizes both Gaussian (image-only) and discrete (text-only) randomized smoothing. We validate the framework on multimodal safety filtering, providing, to our knowledge, the first model-agnostic Neyman--Pearson certificate for joint discrete-token and continuous-image perturbations in interaction-dependent text--image safety filtering.