Large Neighborhood Search meets Iterative Neural Constraint Heuristics

TL;DR

This paper connects neural iterative heuristics with Large Neighborhood Search, adapting a neural constraint satisfaction method into an LNS framework, leading to improved performance on combinatorial problems.

Contribution

It explicitly links neural heuristics with LNS, introduces novel prediction-guided destroy operators, and demonstrates enhanced results on multiple constraint satisfaction problems.

Findings

Neural LNS outperforms vanilla neural heuristics and classical baselines.

Stochastic destroy operators are more effective than greedy ones.

Greedy repair yields better single feasible solutions than sampling-based repair.

Abstract

Neural networks are being increasingly used as heuristics for constraint satisfaction. These neural methods are often recurrent, learning to iteratively refine candidate assignments. In this work, we make explicit the connection between such iterative neural heuristics and Large Neighborhood Search (LNS), and adapt an existing neural constraint satisfaction method-ConsFormer-into an LNS procedure. We decompose the resulting neural LNS into two standard components: the destroy and repair operators. On the destroy side, we instantiate several classical heuristics and introduce novel prediction-guided operators that exploit the model's internal scores to select neighborhoods. On the repair side, we utilize ConsFormer as a neural repair operator and compare the original sampling-based decoder to a greedy decoder that selects the most likely assignments. Through an empirical study on Sudoku, Graph Coloring, and MaxCut, we find that adapting the neural heuristic to an LNS procedure yields substantial gains over its vanilla settings and improves its competitiveness with classical and neural baselines. We further observe consistent design patterns across tasks: stochastic destroy operators outperform greedy ones, while greedy repair is more effective than sampling-based repair for finding a single high-quality feasible assignment. These findings highlight LNS as a useful lens and design framework for structuring and improving iterative neural approaches.