A Survey on Reinforcement Learning for Combinatorial Optimization

TL;DR

This survey reviews reinforcement learning methods applied to combinatorial optimization, focusing on the traveling salesperson problem, highlighting recent deep learning integrations that improve solution quality.

Contribution

It provides a comprehensive comparison of historical and modern RL approaches to TSP, emphasizing the role of deep learning techniques like attention mechanisms.

Findings

Deep RL with attention mechanisms effectively approximates TSP solutions.

Integration of deep learning enhances traditional RL algorithms for combinatorial problems.

Modern RL algorithms outperform earlier methods due to advances in machine learning and computing power.

Abstract

This paper gives a detailed review of reinforcement learning (RL) in combinatorial optimization, introduces the history of combinatorial optimization starting in the 1950s, and compares it with the RL algorithms of recent years. This paper explicitly looks at a famous combinatorial problem-traveling salesperson problem (TSP). It compares the approach of modern RL algorithms for the TSP with an approach published in the 1970s. By comparing the similarities and variances between these methodologies, the paper demonstrates how RL algorithms are optimized due to the evolution of machine learning techniques and computing power. The paper then briefly introduces the deep learning approach to the TSP named deep RL, which is an extension of the traditional mathematical framework. In deep RL, attention and feature encoding mechanisms are introduced to generate near-optimal solutions. The survey shows that integrating the deep learning mechanism, such as attention with RL, can effectively approximate the TSP. The paper also argues that deep learning could be a generic approach that can be integrated with any traditional RL algorithm to enhance the outcomes of the TSP.