Rule-Based Reinforcement Learning for Efficient Robot Navigation with Space Reduction

TL;DR

This paper introduces a rule-based reinforcement learning algorithm for robot navigation that reduces exploration space and accelerates learning, combining classical rules with RL to improve efficiency in complex environments.

Contribution

The paper proposes a novel rule-based RL method that reduces sample complexity and exploration space, with theoretical guarantees and practical validation on real robots.

Findings

Reduced exploration space improves learning efficiency.

Theoretical guarantee that optimal paths are within the reduced space.

Experimental results show enhanced navigation performance.

Abstract

For real-world deployments, it is critical to allow robots to navigate in complex environments autonomously. Traditional methods usually maintain an internal map of the environment, and then design several simple rules, in conjunction with a localization and planning approach, to navigate through the internal map. These approaches often involve a variety of assumptions and prior knowledge. In contrast, recent reinforcement learning (RL) methods can provide a model-free, self-learning mechanism as the robot interacts with an initially unknown environment, but are expensive to deploy in real-world scenarios due to inefficient exploration. In this paper, we focus on efficient navigation with the RL technique and combine the advantages of these two kinds of methods into a rule-based RL (RuRL) algorithm for reducing the sample complexity and cost of time. First, we use the rule of wall-following to generate a closed-loop trajectory. Second, we employ a reduction rule to shrink the trajectory, which in turn effectively reduces the redundant exploration space. Besides, we give the detailed theoretical guarantee that the optimal navigation path is still in the reduced space. Third, in the reduced space, we utilize the Pledge rule to guide the exploration strategy for accelerating the RL process at the early stage. Experiments conducted on real robot navigation problems in hex-grid environments demonstrate that RuRL can achieve improved navigation performance.