Evaluating Program Sequences with Double Machine Learning: An Application to Labor Market Policies

TL;DR

This paper reviews and applies Double Machine Learning methods to evaluate sequential labor market programs, demonstrating how to estimate effects of dynamic policies using Swiss administrative data, and identifying effective interventions.

Contribution

It introduces a framework for program evaluation with sequential structures using DML, extending estimands to dynamic policies and demonstrating empirical application.

Findings

Temporary wage subsidy is most effective intervention.

DML effectively estimates effects of dynamic, multi-period treatments.

Methodology accounts for dynamic confounding in observational data.

Abstract

Many programs evaluated in observational studies incorporate a sequential structure, where individuals may be assigned to various programs over time. While this complexity is often simplified by analyzing programs at single points in time, this paper reviews, explains, and applies methods for program evaluation within a sequential framework. It outlines the assumptions required for identification under dynamic confounding and demonstrates how extending sequential estimands to dynamic policies enables the construction of more realistic counterfactuals. Furthermore, the paper explores recently developed methods for estimating effects across multiple treatments and time periods, utilizing Double Machine Learning (DML), a flexible estimator that avoids parametric assumptions while preserving desirable statistical properties. Using Swiss administrative data, the methods are demonstrated through an empirical application assessing the participation of unemployed individuals in active labor market policies, where assignment decisions by caseworkers can be reconsidered between two periods. The analysis identifies a temporary wage subsidy as the most effective intervention, on average, even after adjusting for its extended duration compared to other programs. Overall, DML-based analysis of dynamic policies proves to be a useful approach within the program evaluation toolkit.