Neural Online Graph Exploration

TL;DR

This paper introduces a reinforcement learning approach to online graph exploration, enabling an agent to learn efficient strategies for discovering unknown graphs, outperforming traditional algorithms on various datasets.

Contribution

First data-driven method for online graph exploration using reinforcement learning with Direct Future Prediction, handling dynamic state and action spaces.

Findings

Agent learns strategies superior to traditional algorithms

Effective on both procedurally generated and real city road networks

Demonstrates exploration can be learned through reinforcement learning

Abstract

Can we learn how to explore unknown spaces efficiently? To answer this question, we study the problem of Online Graph Exploration, the online version of the Traveling Salesperson Problem. We reformulate graph exploration as a reinforcement learning problem and apply Direct Future Prediction (Dosovitskiy and Koltun, 2017) to solve it. As the graph is discovered online, the corresponding Markov Decision Process entails a dynamic state space, namely the observable graph and a dynamic action space, namely the nodes forming the graph's frontier. To the best of our knowledge, this is the first attempt to solve online graph exploration in a data-driven way. We conduct experiments on six data sets of procedurally generated graphs and three real city road networks. We demonstrate that our agent can learn strategies superior to many well known graph traversal algorithms, confirming that exploration can be learned.