Dynamic Treatment on Networks

TL;DR

This paper introduces Q-Ising, a three-stage framework combining Bayesian modeling and reinforcement learning to optimize dynamic treatment strategies in networks, accounting for spillovers and uncertainty.

Contribution

It integrates network dynamics estimation with offline reinforcement learning to develop adaptive treatment policies, a novel approach in network intervention strategies.

Findings

Adaptive targeting outperforms static centrality benchmarks in simulations.

The method provides interpretable spillover estimates and uncertainty quantification.

Finite-sample regret bounds decompose into multiple sources of error.

Abstract

In networks, effective dynamic treatment allocation requires deciding both whom to treat and also when, so as to amplify policy impact through spillovers. An early intervention at a well-connected node can trigger cascades that change which nodes are worth targeting in the next period. Existing treatment strategies under network interference are largely static while dynamic treatment frameworks typically ignore network structure altogether. We integrate these perspectives and propose Q-Ising, a three-stage pipeline that (i) estimates network adoption dynamics via a Bayesian dynamic Ising model from a single observed panel, (ii) augments treatment adoption histories with continuous posterior latent states, and (iii) learns a dynamic policy via offline reinforcement learning. The Bayesian mechanism enables uncertainty quantification over dynamic decisions, yielding posterior ensemble policies with interpretable spillover estimates. We provide a finite-sample regret upper bound that decomposes into standard offline-RL uncertainty, network abstraction error, and first stage error in Ising state estimation. We apply our method to data from Indian village microfinance networks and synthetic stochastic block models under simulated heterogeneous susceptible-infected-susceptible (SIS) dynamics and demonstrate that adaptive targeting outperforms static centrality benchmarks.