Beware of the Classical Benchmark Instances for the Traveling Salesman Problem with Time Windows

TL;DR

This paper introduces an exact, efficient method for solving classical TSPTW-M benchmark instances, revealing their limitations for evaluating new algorithms and highlighting the need for more challenging test sets.

Contribution

An exact algorithm capable of solving large TSPTW-M instances quickly, demonstrating that existing benchmarks are too easy and may mislead algorithm performance assessments.

Findings

All classical benchmarks with 50+ customers solved in under ten seconds.

Most instances solved for the Duration objective using the same method.

Existing benchmarks are not sufficiently challenging for evaluating advanced algorithms.

Abstract

We propose a simple and exact method for the Traveling Salesman Problem with Time Windows and Makespan objective (\TSPTW-M) that solves all instances of the classical benchmark with $50$ or more customers in less than ten seconds each. Applying this algorithm as an off-the-shelf method, we also solve all but one of these instances for the Duration objective. Our main conclusion is that these instances alone are no longer representative for evaluating the TSPTW-M and its Duration variant: their structure can be exploited to yield results that seem outstanding at first glance. Additionally, caution is advised when designing hard training sets for machine learning algorithms.