A Fully Controllable Agent in the Path Planning using Goal-Conditioned Reinforcement Learning

TL;DR

This paper introduces a goal-conditioned reinforcement learning framework with bi-directional memory editing and sub-goal networks, enabling a fully controllable agent to efficiently navigate to various goals in path planning tasks.

Contribution

The study presents a novel reinforcement learning approach that allows an agent to reach multiple goals using bi-directional memory editing and dedicated sub-goal networks, improving flexibility and efficiency.

Findings

Agent successfully reached unseen goals during testing.

The method enabled the agent to perform complex tasks like round trips.

Reward shaping reduced the number of steps to reach goals.

Abstract

The aim of path planning is to reach the goal from starting point by searching for the route of an agent. In the path planning, the routes may vary depending on the number of variables such that it is important for the agent to reach various goals. Numerous studies, however, have dealt with a single goal that is predefined by the user. In the present study, I propose a novel reinforcement learning framework for a fully controllable agent in the path planning. To do this, I propose a bi-directional memory editing to obtain various bi-directional trajectories of the agent, in which the behavior of the agent and sub-goals are trained on the goal-conditioned RL. As for moving the agent in various directions, I utilize the sub-goals dedicated network, separated from a policy network. Lastly, I present the reward shaping to shorten the number of steps for the agent to reach the goal. In the experimental result, the agent was able to reach the various goals that have never been visited by the agent in the training. We confirmed that the agent could perform difficult missions such as a round trip and the agent used the shorter route with the reward shaping.