3D RL-DWA: A Hybrid Reinforcement Learning and Dynamic Window Approach for Goal-Directed Local Navigation in Multi-DoF Robots

TL;DR

This paper introduces a hybrid RL and DWA approach for 3D local navigation of high-DoF robots, improving adaptability and performance in complex environments using sparse data.

Contribution

The novel combination of reinforcement learning with a dynamic window approach for deformable 3D navigation in high-DoF robots.

Findings

Significantly improved deformation and navigation performance over pure RL and model-based methods.

High path completion rate and robustness in unseen scenarios.

Effective navigation using sparse point cloud data in complex environments.

Abstract

In this paper, we present a novel hybrid approach that combines Reinforcement Learning (RL) with Dynamic Window Approach (DWA) for adaptive 3D local navigation of high-degree-of-freedom robotic systems. Our method leverages sparse point cloud data to dynamically adjust both the motion and the shape of a deformable microrobot, enabling the system to navigate toward a goal in complex, constrained environments while maximizing the occupied volume. We evaluate our framework in a simulated vascular network. Experimental results, based on 1080 trials, indicate that integrating RL with a DWA-based local planner significantly enhances both deformation and navigation capabilities compared to a pure RL and a model-based methods. In particular, the proposed autonomous controller consistently achieves high deformation and near-perfect path completion during training and maintains robust performance in unseen scenarios. These findings highlight the potential of hybrid planning strategies for efficient and adaptive 3D navigation under sparse sensory conditions.