Physics-Informed Causal MDPs for Sequential Constraint Repair in Engineering Simulation Pipelines

TL;DR

This paper introduces PI-CMDP, a framework combining causal identification, state-space compression, and physics-guided estimation to improve constraint repair in engineering simulation pipelines, achieving higher success rates with fewer episodes.

Contribution

The paper presents a novel layered DAG-based framework with an Identify-Compress-Estimate pipeline for efficient causal learning and state abstraction in constrained MDPs.

Findings

PI-CMDP achieves 76.2% repair success with 300 episodes.

It narrows the success gap compared to baselines in full-data regime.

Improvements are consistent across multiple seeds with p < 0.02.

Abstract

Off-policy learning in constrained MDPs with large binary state spaces faces a fundamental tension: causal identification of transition dynamics requires structural assumptions, while sample-efficient policy learning requires state-space compression. We introduce PI-CMDP, a framework for CMDPs whose constraint dependencies form a layered DAG under a Lifecycle Ordering Assumption (LOA). We propose an Identify-Compress-Estimate pipeline: (i) Identify: LOA enables backdoor identification of causal edge weights for cross-layer pairs, with formal partial-identification bounds when LOA is violated; (ii) Compress: a Markov abstraction compresses state cardinality from 2^(WL) to (W+1)^L under layer-priority regularity and exchangeability; and (iii) Estimate: a physics-guided doubly-robust estimator remains unbiased and reduces the variance constant when the physics prior outperforms a learned model. We instantiate PI-CMDP on constraint repair in engineering simulation pipelines. On the TPS benchmark (4,206 episodes), PI-CMDP achieves 76.2% repair success rate with only 300 training episodes versus 70.8% for the strongest baseline (+5.4 pp), narrowing to +2.8 pp (83.4% vs. 80.6%) in the full-data regime, while substantially reducing cascade failure rates. All improvements are consistent across 5 independent seeds (paired t-test p < 0.02).