FH-DRL: Exponential-Hyperbolic Frontier Heuristics with DRL for accelerated Exploration in Unknown Environments

TL;DR

FH-DRL introduces a novel exploration framework combining an exponential-hyperbolic frontier heuristic with deep reinforcement learning, significantly improving exploration efficiency in unknown environments through adaptive frontier ranking and real-time environmental assessment.

Contribution

The paper presents FH-DRL, integrating a new exponential-hyperbolic heuristic with DRL for enhanced autonomous exploration, outperforming existing methods in simulated and real-world scenarios.

Findings

Reduces total exploration time and travel distance.

Outperforms standard frontier and DRL-based methods.

Demonstrates robustness in real-world cluttered environments.

Abstract

Autonomous robot exploration in large-scale or cluttered environments remains a central challenge in intelligent vehicle applications, where partial or absent prior maps constrain reliable navigation. This paper introduces FH-DRL, a novel framework that integrates a customizable heuristic function for frontier detection with a Twin Delayed DDPG (TD3) agent for continuous, high-speed local navigation. The proposed heuristic relies on an exponential-hyperbolic distance score, which balances immediate proximity against long-range exploration gains, and an occupancy-based stochastic measure, accounting for environmental openness and obstacle densities in real time. By ranking frontiers using these adaptive metrics, FH-DRL targets highly informative yet tractable waypoints, thereby minimizing redundant paths and total exploration time. We thoroughly evaluate FH-DRL across multiple simulated and real-world scenarios, demonstrating clear improvements in travel distance and completion time over frontier-only or purely DRL-based exploration. In structured corridor layouts and maze-like topologies, our architecture consistently outperforms standard methods such as Nearest Frontier, Cognet Frontier Exploration, and Goal Driven Autonomous Exploration. Real-world tests with a Turtlebot3 platform further confirm robust adaptation to previously unseen or cluttered indoor spaces. The results highlight FH-DRL as an efficient and generalizable approach for frontier-based exploration in large or partially known environments, offering a promising direction for various autonomous driving, industrial, and service robotics tasks.