Hybrid DQN-TD3 Reinforcement Learning for Autonomous Navigation in Dynamic Environments

TL;DR

This paper introduces a hierarchical reinforcement learning framework combining DQN and TD3 for autonomous navigation, demonstrating improved success, safety, and efficiency in dynamic environments.

Contribution

It presents a novel hybrid DQN-TD3 architecture with a safety gate and reward shaping for robust navigation in dynamic, partially observable settings.

Findings

Higher success rate compared to baseline algorithms

Enhanced generalization to unseen obstacle scenarios

Reduced abrupt control changes and improved safety

Abstract

This paper presents a hierarchical path-planning and control framework that combines a high-level Deep Q-Network (DQN) for discrete sub-goal selection with a low-level Twin Delayed Deep Deterministic Policy Gradient (TD3) controller for continuous actuation. The high-level module selects behaviors and sub-goals; the low-level module executes smooth velocity commands. We design a practical reward shaping scheme (direction, distance, obstacle avoidance, action smoothness, collision penalty, time penalty, and progress), together with a LiDAR-based safety gate that prevents unsafe motions. The system is implemented in ROS + Gazebo (TurtleBot3) and evaluated with PathBench metrics, including success rate, collision rate, path efficiency, and re-planning efficiency, in dynamic and partially observable environments. Experiments show improved success rate and sample efficiency over single-algorithm baselines (DQN or TD3 alone) and rule-based planners, with better generalization to unseen obstacle configurations and reduced abrupt control changes. Code and evaluation scripts are available at the project repository.