Transferable Adversarial Robustness for Categorical Data via Universal Robust Embeddings

TL;DR

This paper introduces a method to enhance adversarial robustness in tabular data by creating universal robust embeddings, enabling transfer to various classifiers without retraining, thus improving security in critical domains.

Contribution

The paper proposes a novel bilevel optimization framework for universal robust embeddings tailored to categorical data, facilitating robustness transfer to non-neural classifiers.

Findings

Outperforms existing robustness techniques on tabular data

Enables robustness transfer to boosted trees and random forests

Maintains high accuracy while improving adversarial resilience

Abstract

Research on adversarial robustness is primarily focused on image and text data. Yet, many scenarios in which lack of robustness can result in serious risks, such as fraud detection, medical diagnosis, or recommender systems often do not rely on images or text but instead on tabular data. Adversarial robustness in tabular data poses two serious challenges. First, tabular datasets often contain categorical features, and therefore cannot be tackled directly with existing optimization procedures. Second, in the tabular domain, algorithms that are not based on deep networks are widely used and offer great performance, but algorithms to enhance robustness are tailored to neural networks (e.g. adversarial training). In this paper, we tackle both challenges. We present a method that allows us to train adversarially robust deep networks for tabular data and to transfer this robustness to other classifiers via universal robust embeddings tailored to categorical data. These embeddings, created using a bilevel alternating minimization framework, can be transferred to boosted trees or random forests making them robust without the need for adversarial training while preserving their high accuracy on tabular data. We show that our methods outperform existing techniques within a practical threat model suitable for tabular data.