Learning Generative Dynamics with Soft Law Constraints: A McKean-Vlasov FBSDE Approach

TL;DR

This paper introduces a novel generative framework using McKean-Vlasov FBSDEs to learn stochastic dynamics from distributional data, with applications in image and human motion modeling.

Contribution

It develops a new neural solver for coupled FBSDEs enforcing soft law constraints, enabling learning of complex stochastic paths from distributional observations.

Findings

Successfully recovers smooth stochastic paths matching target distributions.

Demonstrates effective transport of facial expressions in latent space.

Produces coherent human motion trajectories from high-dimensional pose data.

Abstract

We propose a generative framework for learning stochastic dynamics from endpoint and intermediate distributional observations. The method formulates generation as a McKean-Vlasov control problem in which terminal and time-marginal laws are enforced through soft energy constraints. The associated optimality system is a forward-backward stochastic differential equation (FBSDE) whose backward component receives a continuous drift induced by the marginal law penalties. This provides a principled alternative to hard interpolation or optimal transport maps between observed distributions: the model learns a stochastic path law whose dynamics remain globally coupled through the mean-field objective. We derive the reduced FBSDE system for quadratic control cost and constant diffusion, connecting terminal and marginal law flat derivatives to score-like training signals. The resulting neural solver is evaluated on low-dimensional distributional benchmarks, where it recovers smooth stochastic paths matching prescribed marginal laws. In a higher-dimensional ALAE latent space, endpoint supervision is used as a qualitative stress test for transporting non-smiling faces toward smiling ones in a pretrained representation. We then use articulated human motion as a structured high-dimensional case study on a curated AMASS low-to-high position dataset, using SMPL-H pose sequences and reduced pose representations. The experiments show that soft marginal law constraints can produce coherent stochastic trajectories whose intermediate distributions follow the observed evolution of human motion. The code is available at https://github.com/murex/deep-mkv-gen/tree/main.