Multi-CALF: A Policy Combination Approach with Statistical Guarantees

TL;DR

Multi-CALF is a novel algorithm that combines reinforcement learning policies with statistical guarantees, ensuring stability and improved performance in control tasks.

Contribution

It introduces a policy combination method with formal convergence guarantees and empirical validation for enhanced control performance.

Findings

Achieves better control performance than individual policies.

Provides formal convergence and stability guarantees.

Demonstrates effectiveness on control tasks.

Abstract

We introduce Multi-CALF, an algorithm that intelligently combines reinforcement learning policies based on their relative value improvements. Our approach integrates a standard RL policy with a theoretically-backed alternative policy, inheriting formal stability guarantees while often achieving better performance than either policy individually. We prove that our combined policy converges to a specified goal set with known probability and provide precise bounds on maximum deviation and convergence time. Empirical validation on control tasks demonstrates enhanced performance while maintaining stability guarantees.