Efficient Decoder Scaling Strategy for Neural Routing Solvers

TL;DR

This paper systematically investigates how scaling depth versus width in neural routing decoders affects performance, revealing that increasing depth is more beneficial than width for vehicle routing problems.

Contribution

It introduces a comprehensive analysis of decoder scaling strategies, demonstrating the advantages of depth scaling over width scaling in neural routing models.

Findings

Scaling depth improves performance more than scaling width.

Parameter count alone does not predict model effectiveness.

Design principles for efficient model scaling are validated.

Abstract

Construction-based neural routing solvers, typically composed of an encoder and a decoder, have emerged as a promising approach for solving vehicle routing problems. While recent studies suggest that shifting parameters from the encoder to the decoder enhances performance, most works restrict the decoder size to 1-3M parameters, leaving the effects of scaling largely unexplored. To address this gap, we conduct a systematic study comparing two distinct strategies: scaling depth versus scaling width. We synthesize these strategies to construct a suite of 12 model configurations, spanning a parameter range from 1M to ~150M, and extensively evaluate their scaling behaviors across three critical dimensions: parameter efficiency, data efficiency, and compute efficiency. Our empirical results reveal that parameter count is insufficient to accurately predict the model performance, highlighting the critical and distinct roles of model depth (layer count) and width (embedding dimension). Crucially, we demonstrate that scaling depth yields superior performance gains to scaling width. Based on these findings, we provide and experimentally validate a set of design principles for the efficient allocation of parameters and compute resources to enhance the model performance.