Synergizing Reinforcement Learning and Genetic Algorithms for Neural Combinatorial Optimization

TL;DR

This paper introduces a novel framework that combines deep reinforcement learning with genetic algorithms to improve the efficiency and quality of solutions for complex combinatorial optimization problems.

Contribution

The paper proposes the Evolutionary Augmentation Mechanism (EAM), a versatile framework that enhances DRL with genetic algorithm techniques for better exploration and convergence.

Findings

EAM improves solution quality on TSP, CVRP, PCTSP, and OP benchmarks.

EAM accelerates training convergence compared to baseline methods.

EAM is compatible with multiple state-of-the-art DRL solvers.

Abstract

Combinatorial optimization problems are notoriously challenging due to their discrete structure and exponentially large solution space. Recent advances in deep reinforcement learning (DRL) have enabled the learning heuristics directly from data. However, DRL methods often suffer from limited exploration and susceptibility to local optima. On the other hand, evolutionary algorithms such as Genetic Algorithms (GAs) exhibit strong global exploration capabilities but are typically sample inefficient and computationally intensive. In this work, we propose the Evolutionary Augmentation Mechanism (EAM), a general and plug-and-play framework that synergizes the learning efficiency of DRL with the global search power of GAs. EAM operates by generating solutions from a learned policy and refining them through domain-specific genetic operations such as crossover and mutation. These evolved solutions are then selectively reinjected into the policy training loop, thereby enhancing exploration and accelerating convergence. We further provide a theoretical analysis that establishes an upper bound on the KL divergence between the evolved solution distribution and the policy distribution, ensuring stable and effective policy updates. EAM is model-agnostic and can be seamlessly integrated with state-of-the-art DRL solvers such as the Attention Model, POMO, and SymNCO. Extensive results on benchmark problems (e.g., TSP, CVRP, PCTSP, and OP) demonstrate that EAM significantly improves both solution quality and training efficiency over competitive baselines.