Unrealized Expectations: Comparing AI Methods vs Classical Algorithms for Maximum Independent Set

TL;DR

This paper demonstrates that classical algorithms outperform AI-inspired methods for the Maximum Independent Set problem, highlighting the need for better benchmarking and integration of classical heuristics in AI approaches.

Contribution

The paper provides a comprehensive comparison between AI methods and classical algorithms for MIS, introducing a novel analysis technique called serialization to explain AI failures.

Findings

Classical solver KaMIS outperforms AI-inspired methods on random graphs.

AI methods often fail to surpass simple greedy heuristics.

Serialization analysis shows AI methods reason similarly to basic heuristics.

Abstract

AI methods, such as generative models and reinforcement learning, have recently been applied to combinatorial optimization (CO) problems, especially NP-hard ones. This paper compares such GPU-based methods with classical CPU-based methods on the Maximum Independent Set (MIS) problem. Strikingly, even on in-distribution random graphs, leading AI-inspired methods are consistently outperformed by the state-of-the-art classical solver KaMIS running on a single CPU, and some AI-inspired methods frequently fail to surpass even the simplest degree-based greedy heuristic. Even with post-processing techniques like local search, AI-inspired methods still perform worse than CPU-based solvers. To better understand the source of these failures, we introduce a novel analysis, serialization, which reveals that non-backtracking AI-inspired methods, e.g. LTFT (which is based on GFlowNets), end up reasoning similarly to the simplest degree-based greedy, and thus worse than KaMIS. More generally, our findings suggest a need for a rethinking of current approaches in AI for CO, advocating for more rigorous benchmarking and the principled integration of classical heuristics. Additionally, we also find that CPU-based algorithm KaMIS have strong performance on sparse random graphs, which appears to show that the shattering threshold conjecture for large independent sets proposed by Coja-Oghlan & Efthymiou (2015) does not apply for real-life sizes (such as 10^6 nodes).