Evaluating Black-Box Vulnerabilities with Wasserstein-Constrained Data Perturbations

TL;DR

This paper introduces a model-agnostic framework using Wasserstein constraints to analyze ML robustness against realistic data perturbations, enhancing explainability and fairness diagnostics.

Contribution

It presents a novel approach combining Optimal Transport and Distributionally Robust Optimization for realistic, feature-level constrained data perturbations.

Findings

Provides a theoretical guarantee for the robustness diagnostics.

Validates the approach on real-world datasets in tabular and image domains.

Offers a diagnostic tool that complements existing evaluation methods.

Abstract

The growing use of Machine Learning (ML) tools comes with critical challenges, such as limited model explainability. We propose a global explainability framework that leverages Optimal Transport and Distributionally Robust Optimization to analyze how ML algorithms respond to constrained data perturbations. Our approach enforces constraints on feature-level statistics (e.g., brightness, age distribution), generating realistic perturbations that preserve semantic structure. We provide a model-agnostic diagnostic bench that applies to both tabular and image domains with solid theoretical guarantees. We validate the approach on real-world datasets providing interpretable robustness diagnostics that complement standard evaluation and fairness auditing tools.