Learning Generalizable Models for Vehicle Routing Problems via Knowledge Distillation

TL;DR

This paper introduces an Adaptive Multi-Distribution Knowledge Distillation scheme to improve the cross-distribution generalization of neural models for vehicle routing problems, achieving competitive results with less computation.

Contribution

It proposes a novel knowledge distillation method that leverages multiple teachers and an adaptive strategy to enhance model generalization across diverse distributions.

Findings

Achieves competitive performance on unseen distributions

Outperforms baseline neural methods in generalization

Consumes less computational resources during inference

Abstract

Recent neural methods for vehicle routing problems always train and test the deep models on the same instance distribution (i.e., uniform). To tackle the consequent cross-distribution generalization concerns, we bring the knowledge distillation to this field and propose an Adaptive Multi-Distribution Knowledge Distillation (AMDKD) scheme for learning more generalizable deep models. Particularly, our AMDKD leverages various knowledge from multiple teachers trained on exemplar distributions to yield a light-weight yet generalist student model. Meanwhile, we equip AMDKD with an adaptive strategy that allows the student to concentrate on difficult distributions, so as to absorb hard-to-master knowledge more effectively. Extensive experimental results show that, compared with the baseline neural methods, our AMDKD is able to achieve competitive results on both unseen in-distribution and out-of-distribution instances, which are either randomly synthesized or adopted from benchmark datasets (i.e., TSPLIB and CVRPLIB). Notably, our AMDKD is generic, and consumes less computational resources for inference.