The Robot Routing Problem for Collecting Aggregate Stochastic Rewards

TL;DR

This paper introduces a stochastic graph model for reward collection where rewards appear and disappear randomly, and studies the complexity of finding optimal paths for a robot to maximize expected rewards over finite and infinite horizons.

Contribution

It formalizes the robot routing problem with stochastic rewards, analyzes its computational complexity, and provides algorithms for epsilon-optimal path computation.

Findings

NP-hardness of the problem

Optimal strategies may require memory

Algorithm for epsilon-optimal infinite paths

Abstract

We propose a new model for formalizing reward collection problems on graphs with dynamically generated rewards which may appear and disappear based on a stochastic model. The *robot routing problem* is modeled as a graph whose nodes are stochastic processes generating potential rewards over discrete time. The rewards are generated according to the stochastic process, but at each step, an existing reward disappears with a given probability. The edges in the graph encode the (unit-distance) paths between the rewards' locations. On visiting a node, the robot collects the accumulated reward at the node at that time, but traveling between the nodes takes time. The optimization question asks to compute an optimal (or epsilon-optimal) path that maximizes the expected collected rewards. We consider the finite and infinite-horizon robot routing problems. For finite-horizon, the goal is to maximize the total expected reward, while for infinite horizon we consider limit-average objectives. We study the computational and strategy complexity of these problems, establish NP-lower bounds and show that optimal strategies require memory in general. We also provide an algorithm for computing epsilon-optimal infinite paths for arbitrary epsilon > 0.