Fast Algorithms for Stackelberg Prediction Game with Least Squares Loss

TL;DR

This paper introduces a fast, scalable algorithm for solving Stackelberg prediction games with least squares loss by reformulating the problem as a second order cone program, enabling real-time solutions for large datasets.

Contribution

The authors propose a novel SDP reformulation of SPG-LS that is significantly faster than existing methods and can be reduced to an SOCP for real-time applications.

Findings

The SOCP approach is up to 20,000+ times faster than previous methods.

The reformulation enables real-time solutions for large-scale SPG-LS problems.

Numerical experiments confirm the efficiency and scalability of the proposed method.

Abstract

The Stackelberg prediction game (SPG) has been extensively used to model the interactions between the learner and data provider in the training process of various machine learning algorithms. Particularly, SPGs played prominent roles in cybersecurity applications, such as intrusion detection, banking fraud detection, spam filtering, and malware detection. Often formulated as NP-hard bi-level optimization problems, it is generally computationally intractable to find global solutions to SPGs. As an interesting progress in this area, a special class of SPGs with the least squares loss (SPG-LS) have recently been shown polynomially solvable by a bisection method. However, in each iteration of this method, a semidefinite program (SDP) needs to be solved. The resulted high computational costs prevent its applications for large-scale problems. In contrast, we propose a novel approach that reformulates a SPG-LS as a single SDP of a similar form and the same dimension as those solved in the bisection method. Our SDP reformulation is, evidenced by our numerical experiments, orders of magnitude faster than the existing bisection method. We further show that the obtained SDP can be reduced to a second order cone program (SOCP). This allows us to provide real-time response to large-scale SPG-LS problems. Numerical results on both synthetic and real world datasets indicate that the proposed SOCP method is up to 20,000+ times faster than the state of the art.