A General Framework for Evaluating Robustness of Combinatorial Optimization Solvers on Graphs

TL;DR

This paper introduces a practical robustness metric for combinatorial optimization solvers on graphs, enabling assessment of solver sensitivity without requiring optimal solutions, and reveals significant performance degradation under adversarial conditions.

Contribution

It develops the first feasible robustness metric for general CO solvers that does not depend on optimal solutions and employs black-box adversarial methods for non-differentiable solvers.

Findings

State-of-the-art solvers like Gurobi can lose over 20% performance under adversarial conditions.

The proposed metric effectively identifies instances where solvers are vulnerable.

Robustness concerns are raised for practical applications of CO solvers.

Abstract

Solving combinatorial optimization (CO) on graphs is among the fundamental tasks for upper-stream applications in data mining, machine learning and operations research. Despite the inherent NP-hard challenge for CO, heuristics, branch-and-bound, learning-based solvers are developed to tackle CO problems as accurately as possible given limited time budgets. However, a practical metric for the sensitivity of CO solvers remains largely unexplored. Existing theoretical metrics require the optimal solution which is infeasible, and the gradient-based adversarial attack metric from deep learning is not compatible with non-learning solvers that are usually non-differentiable. In this paper, we develop the first practically feasible robustness metric for general combinatorial optimization solvers. We develop a no worse optimal cost guarantee thus do not require optimal solutions, and we tackle the non-differentiable challenge by resorting to black-box adversarial attack methods. Extensive experiments are conducted on 14 unique combinations of solvers and CO problems, and we demonstrate that the performance of state-of-the-art solvers like Gurobi can degenerate by over 20% under the given time limit bound on the hard instances discovered by our robustness metric, raising concerns about the robustness of combinatorial optimization solvers.