A Geometric Nash Approach in Tuning the Learning Rate in Q-Learning Algorithm

TL;DR

This paper introduces a geometric Nash approach to optimize the learning rate in Q-learning by analyzing the relationship between total time steps and reward vectors, improving learning stability and efficiency.

Contribution

It presents a novel geometric framework using Nash Equilibrium and angular bisectors to systematically estimate the learning rate in Q-learning.

Findings

Relationship between learning rate and angle between T and R vectors

Angular bisector concept aids in estimating optimal alpha

Enhanced stability and efficiency in Q-learning

Abstract

This paper proposes a geometric approach for estimating the $\alpha$ value in Q learning. We establish a systematic framework that optimizes the {\alpha} parameter, thereby enhancing learning efficiency and stability. Our results show that there is a relationship between the learning rate and the angle between a vector T (total time steps in each episode of learning) and R (the reward vector for each episode). The concept of angular bisector between vectors T and R and Nash Equilibrium provide insight into estimating $\alpha$ such that the algorithm minimizes losses arising from exploration-exploitation trade-off.