Adaptive Path-Planning for Autonomous Robots: A UCH-Enhanced Q-Learning Approach

TL;DR

This paper introduces an improved Q-learning framework with adaptive initialization and reward tuning mechanisms, significantly enhancing autonomous robot path-planning efficiency and accuracy through extensive experimental validation.

Contribution

The paper presents a novel IQL framework combining PACO and UCH mechanisms, advancing Q-learning for more efficient and effective robot path planning.

Findings

IQL outperforms existing algorithms like FIQL and QMABC in path-planning tasks.

PACO improves Q-table initialization, accelerating convergence.

UCH enhances reward function tuning, increasing path-planning accuracy.

Abstract

Q-learning methods are widely used in robot path planning but often face challenges of inefficient search and slow convergence. We propose an Improved Q-learning (IQL) framework that enhances standard Q-learning in two significant ways. First, we introduce the Path Adaptive Collaborative Optimization (PACO) algorithm to optimize Q-table initialization, providing better initial estimates and accelerating learning. Second, we incorporate a Utility-Controlled Heuristic (UCH) mechanism with dynamically tuned parameters to optimize the reward function, enhancing the algorithm's accuracy and effectiveness in path-planning tasks. Extensive experiments in three different raster grid environments validate the superior performance of our IQL framework. The results demonstrate that our IQL algorithm outperforms existing methods, including FIQL, PP-QL-based CPP, DFQL, and QMABC algorithms, in terms of path-planning capabilities.