Eyeballing Combinatorial Problems: A Case Study of Using Multimodal Large Language Models to Solve Traveling Salesman Problems

TL;DR

This paper explores using multimodal large language models' visual capabilities to visually estimate solutions for the Traveling Salesman Problem, demonstrating promising results in zero-shot and few-shot scenarios.

Contribution

It introduces a novel approach of leveraging MLLMs' visual reasoning to 'eyeball' solutions for combinatorial problems like TSP, with minimal training.

Findings

MLLMs can effectively 'eyeball' TSP routes from images.

Zero-shot and few-shot evaluations yield promising results.

Self-ensemble and self-refine methods improve solution quality.

Abstract

Multimodal Large Language Models (MLLMs) have demonstrated proficiency in processing di-verse modalities, including text, images, and audio. These models leverage extensive pre-existing knowledge, enabling them to address complex problems with minimal to no specific training examples, as evidenced in few-shot and zero-shot in-context learning scenarios. This paper investigates the use of MLLMs' visual capabilities to 'eyeball' solutions for the Traveling Salesman Problem (TSP) by analyzing images of point distributions on a two-dimensional plane. Our experiments aimed to validate the hypothesis that MLLMs can effectively 'eyeball' viable TSP routes. The results from zero-shot, few-shot, self-ensemble, and self-refine zero-shot evaluations show promising outcomes. We anticipate that these findings will inspire further exploration into MLLMs' visual reasoning abilities to tackle other combinatorial problems.