Off-Policy Evaluation in Embedded Spaces
Jaron J. R. Lee, David Arbour, Georgios Theocharous

TL;DR
This paper introduces a new off-policy evaluation method that uses embedded action spaces and normalizing flows to address positivity violations and feasibility issues in large, non-probabilistic recommendation systems.
Contribution
It proposes the featurized embedded permutation weighting estimator, combining action embeddings and density ratio estimation via normalizing flows for improved off-policy evaluation.
Findings
Reduces positivity violations in large action spaces.
Enables density ratio estimation without explicit policy probabilities.
Demonstrates practical feasibility with recent density estimation techniques.
Abstract
Off-policy evaluation methods are important in recommendation systems and search engines, where data collected under an existing logging policy is used to estimate the performance of a new proposed policy. A common approach to this problem is weighting, where data is weighted by a density ratio between the probability of actions given contexts in the target and logged policies. In practice, two issues often arise. First, many problems have very large action spaces and we may not observe rewards for most actions, and so in finite samples we may encounter a positivity violation. Second, many recommendation systems are not probabilistic and so having access to logging and target policy densities may not be feasible. To address these issues, we introduce the featurized embedded permutation weighting estimator. The estimator computes the density ratio in an action embedding space, which…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsPrivacy-Preserving Technologies in Data · Topic Modeling · Access Control and Trust
MethodsNormalizing Flows
