Learning Coverage Paths in Unknown Environments with Deep Reinforcement Learning

TL;DR

This paper explores using deep reinforcement learning to develop online coverage path planning strategies for unknown environments, emphasizing a novel map representation and reward function to improve coverage efficiency.

Contribution

It introduces a new egocentric frontier-based map representation and a total variation reward term, advancing RL methods for coverage path planning in unknown spaces.

Findings

Outperforms previous RL-based methods in coverage tasks

Achieves better coverage efficiency across multiple environment types

Demonstrates computational feasibility for real-time applications

Abstract

Coverage path planning (CPP) is the problem of finding a path that covers the entire free space of a confined area, with applications ranging from robotic lawn mowing to search-and-rescue. When the environment is unknown, the path needs to be planned online while mapping the environment, which cannot be addressed by offline planning methods that do not allow for a flexible path space. We investigate how suitable reinforcement learning is for this challenging problem, and analyze the involved components required to efficiently learn coverage paths, such as action space, input feature representation, neural network architecture, and reward function. We propose a computationally feasible egocentric map representation based on frontiers, and a novel reward term based on total variation to promote complete coverage. Through extensive experiments, we show that our approach surpasses the performance of both previous RL-based approaches and highly specialized methods across multiple CPP variations.