Optimism in Reinforcement Learning with Generalized Linear Function Approximation

TL;DR

This paper introduces a new efficient reinforcement learning algorithm using generalized linear function approximation, leveraging a weaker assumption called 'optimistic closure' to achieve strong regret bounds.

Contribution

It presents the first statistically and computationally efficient RL algorithm for generalized linear functions under the optimistic closure assumption.

Findings

Achieves a regret bound of O(^3 T)

Introduces the optimistic closure assumption

First efficient algorithm for this setting

Abstract

We design a new provably efficient algorithm for episodic reinforcement learning with generalized linear function approximation. We analyze the algorithm under a new expressivity assumption that we call "optimistic closure," which is strictly weaker than assumptions from prior analyses for the linear setting. With optimistic closure, we prove that our algorithm enjoys a regret bound of $\tilde{O}(\sqrt{d^3 T})$ where $d$ is the dimensionality of the state-action features and $T$ is the number of episodes. This is the first statistically and computationally efficient algorithm for reinforcement learning with generalized linear functions.