A Benchmark for Maximum Cut: Towards Standardization of the Evaluation of Learned Heuristics for Combinatorial Optimization

TL;DR

This paper introduces MaxCut-Bench, a standardized benchmark suite for evaluating heuristics on the Maximum Cut problem, and systematically compares various learned and traditional heuristics, revealing limitations of current learning-based methods.

Contribution

The paper presents a comprehensive benchmark suite for Maximum Cut and evaluates existing heuristics, highlighting the underperformance of many learned heuristics compared to simple algorithms.

Findings

Many learned heuristics do not outperform naive greedy algorithms.

Only one learned heuristic consistently beats Tabu Search.

Replacing GNNs with linear regression can improve ECO-DQN performance.

Abstract

Recently, there has been much work on the design of general heuristics for graph-based, combinatorial optimization problems via the incorporation of Graph Neural Networks (GNNs) to learn distribution-specific solution structures.However, there is a lack of consistency in the evaluation of these heuristics, in terms of the baselines and instances chosen, which makes it difficult to assess the relative performance of the algorithms. In this paper, we propose an open-source benchmark suite MaxCut-Bench dedicated to the NP-hard Maximum Cut problem in both its weighted and unweighted variants, based on a careful selection of instances curated from diverse graph datasets. The suite offers a unified interface to various heuristics, both traditional and machine learning-based. Next, we use the benchmark in an attempt to systematically corroborate or reproduce the results of several, popular learning-based approaches, including S2V-DQN [31], ECO-DQN [4], among others, in terms of three dimensions: objective value, generalization, and scalability. Our empirical results show that several of the learned heuristics fail to outperform a naive greedy algorithm, and that only one of them consistently outperforms Tabu Search, a simple, general heuristic based upon local search. Furthermore, we find that the performance of ECO-DQN remains the same or is improved if the GNN is replaced by a simple linear regression on a subset of the features that are related to Tabu Search. Code, data, and pretrained models are available at: \url{https://github.com/ankurnath/MaxCut-Bench}.