Comments on the Du-Kakade-Wang-Yang Lower Bounds

TL;DR

This paper compares and reconciles different theoretical results on the sample complexity of reinforcement learning, focusing on lower bounds and the role of the eluder dimension in problem tractability.

Contribution

It clarifies the relationship between recent lower bounds and the eluder dimension framework, providing a unified understanding of problem complexity in reinforcement learning.

Findings

Reconciles interpretations of lower bounds and eluder dimension.

Highlights conditions under which RL problems are tractable or intractable.

Provides insights into the theoretical landscape of sample complexity in RL.

Abstract

Du, Kakade, Wang, and Yang recently established intriguing lower bounds on sample complexity, which suggest that reinforcement learning with a misspecified representation is intractable. Another line of work, which centers around a statistic called the eluder dimension, establishes tractability of problems similar to those considered in the Du-Kakade-Wang-Yang paper. We compare these results and reconcile interpretations.