Fixed Point Theory Analysis of a Lambda Policy Iteration with Randomization for the \'Ciri\'c Contraction Operator

TL;DR

This paper uses fixed point theory to analyze a randomized Lambda policy iteration algorithm for weak contraction mappings, broadening the scope beyond traditional strong contractions in reinforcement learning.

Contribution

It extends fixed point analysis to weak contraction mappings in policy iteration, providing convergence conditions in infinite-dimensional spaces.

Findings

Convergence with probability one under general assumptions

Applicable to broader class of mappings than traditional contractions

Provides theoretical foundation for reinforcement learning algorithms

Abstract

We apply methods of the fixed point theory to a Lambda policy iteration with a randomization algorithm for weak contractions mappings. This type of mappings covers a broader range than the strong contractions typically considered in the literature, such as \'Ciri\'c contraction. Specifically, we explore the characteristics of reinforcement learning procedures developed for feedback control within the context of fixed point theory. Under relatively general assumptions, we identify the sufficient conditions for convergence with a probability of one in infinite-dimensional policy spaces.