Generating Poisoning Attacks against Ridge Regression Models with Categorical Features

TL;DR

This paper introduces a novel algorithm to generate effective poisoning attacks on ridge regression models with both numerical and categorical features, enhancing understanding of model vulnerabilities.

Contribution

It formulates the poisoning attack as a bilevel optimization problem and proposes a new algorithm specifically targeting categorical features modeled as SOS-1 sets.

Findings

Improves mean squared error across datasets

Outperforms previous benchmarks

Effectively poisons models with categorical features

Abstract

Machine Learning (ML) models have become a very powerful tool to extract information from large datasets and use it to make accurate predictions and automated decisions. However, ML models can be vulnerable to external attacks, causing them to underperform or deviate from their expected tasks. One way to attack ML models is by injecting malicious data to mislead the algorithm during the training phase, which is referred to as a poisoning attack. We can prepare for such situations by designing anticipated attacks, which are later used for creating and testing defence strategies. In this paper, we propose an algorithm to generate strong poisoning attacks for a ridge regression model containing both numerical and categorical features that explicitly models and poisons categorical features. We model categorical features as SOS-1 sets and formulate the problem of designing poisoning attacks as a bilevel optimization problem that is nonconvex mixed-integer in the upper-level and unconstrained convex quadratic in the lower-level. We present the mathematical formulation of the problem, introduce a single-level reformulation based on the Karush-Kuhn-Tucker (KKT) conditions of the lower level, find bounds for the lower-level variables to accelerate solver performance, and propose a new algorithm to poison categorical features. Numerical experiments show that our method improves the mean squared error of all datasets compared to the previous benchmark in the literature.