Regularized Langevin Dynamics for Combinatorial Optimization

TL;DR

This paper introduces Regularized Langevin Dynamics (RLD), a novel sampling framework for combinatorial optimization that improves exploration and efficiency, outperforming state-of-the-art methods in multiple classic problems.

Contribution

The paper proposes RLD, a new regularization technique for Langevin dynamics, and develops two solvers based on RLD, significantly enhancing exploration and efficiency in combinatorial optimization.

Findings

RLD-based simulated annealing reduces runtime by up to 80%.

Both RLD-based methods outperform previous state-of-the-art solvers.

Empirical results on three classic problems demonstrate superior performance.

Abstract

This work proposes a simple yet effective sampling framework for combinatorial optimization (CO). Our method builds on discrete Langevin dynamics (LD), an efficient gradient-guided generative paradigm. However, we observe that directly applying LD often leads to limited exploration. To overcome this limitation, we propose the Regularized Langevin Dynamics (RLD), which enforces an expected distance between the sampled and current solutions, effectively avoiding local minima. We develop two CO solvers on top of RLD, one based on simulated annealing (SA), and the other one based on neural network (NN). Empirical results on three classic CO problems demonstrate that both of our methods can achieve comparable or better performance against the previous state-of-the-art (SOTA) SA- and NN-based solvers. In particular, our SA algorithm reduces the runtime of the previous SOTA SA method by up to 80\%, while achieving equal or superior performance. In summary, RLD offers a promising framework for enhancing both traditional heuristics and NN models to solve CO problems. Our code is available at https://github.com/Shengyu-Feng/RLD4CO.