Policy Learning with the polle package

TL;DR

The polle package in R provides a comprehensive framework for learning and evaluating policies from observational data, integrating multiple methods and ensuring valid inference with flexible machine learning techniques.

Contribution

It introduces a unified R package that combines existing and novel causal policy learning methods with a simple, reproducible workflow.

Findings

Supports various policy learning methods including doubly robust Q-learning and policy trees.

Handles positivity violations by focusing on realistic policies.

Enables valid inference through cross-fitting and flexible ML models.

Abstract

The R package polle is a unifying framework for learning and evaluating finite stage policies based on observational data. The package implements a collection of existing and novel methods for causal policy learning including doubly robust restricted Q-learning, policy tree learning, and outcome weighted learning. The package deals with (near) positivity violations by only considering realistic policies. Highly flexible machine learning methods can be used to estimate the nuisance components and valid inference for the policy value is ensured via cross-fitting. The library is built up around a simple syntax with four main functions policy_data(), policy_def(), policy_learn(), and policy_eval() used to specify the data structure, define user-specified policies, specify policy learning methods and evaluate (learned) policies. The functionality of the package is illustrated via extensive reproducible examples.